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12 10 Steps to SO 
2005 Security 


-John Howie 

Apply protection at the perimeter, network, host, 
application, data, and physical environment layers 
of your database installation. 
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Don’t Be Afraid of BLOBS and CLOBs 


-Yoel Martinez 

With SQL Server 2005’ three new LOB data types and CLR function- 
ality, you can easily compress and decompress large amounts of data. 
InstantDoc ID 95185 


Excel 2007's Powerful PivotTables 


—Tyler Chessman 

Microsoft Excel 2007 PivotTables give you more ways than ever to analyze 
and present data from SQL Server Analysis Services 2005 OLAP cubes. 
InstantDoc ID 94270 


Moving Cubes from Analysis Services 
2000 to 2005 


—Erin Welker 

Deciding to upgrade from Analysis Services 2000 to 2005 might be a 
no-brainer, but figuring out how to move your cubes can be a more 
complicated process. Here are some key concepts to help you decide 
whether to migrate or rebuild your cubes. 
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es Samples 

QL Server 2005 Analysis Services 
samples perform functions that aren’t 
otherwise available in SQL Server 
Management Studio (SSMS) or the SQL 
Server Business Intelligence Development — 
Studio (BIDS). Here are several that you = 
should add to your toolkit. f 
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DATETIME Calculations, Part 3 

—ltzik Ben-Gan 

This third part of a series about datetime 
challenges and calculations focuses on 
techniques for calculating the date of the 
last/next occurrences of a weekday that 
correspond to a given event date. 
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Use Missing-Index Groups for Query 
Tuning 

—Kalen Delaney 

Identify and implement missing-index 
groups to tune your queries, and use 

SQL Server 2005’s XML Showplan output 
feature to examine detailed missing-index 
information. 
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SQL Server Performance Monitoring and 
Management Tools 

—John Green 

Both Idera’s SQL diagnostic manager and Quest 
Software’s Spotlight on SQL Server Enterprise can help 
you quickly detect and diagnose problems with your 
SQL servers. 
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Bi: Pass the 


Editorial 


S Q Servers business intelligence 
(BI) features are part of what 
sets it apart from its competitors. BI is defi- 
nitely not just Microsoft hype. The value in 
BI technologies is how they help businesses 
obtain more information from their data. 
Knowing such things as who your 
customers are, their habits and tastes, what 
they've bought in the past, and how often 
they buy can give you a competitive advan- 
tage. But if you don’t have this kind of 
information, you dont know what you're 
missing. Even though BI support has been 
an integral part of SQL Server for almost 
10 years, the adoption of BI is still far from 
pervasive. In many cases, organizations and 
database professionals don’t see the value in 
BI and think they get along fine without it. 
Results from SQL Server Magazine 
Instant Polls and research from Windows 
IT Pro’s Information Technology Customer 
Research (ITCR) panel show that the 
adoption of BI technologies is still—at 
best—a future goal for many organizations. 
Only 22.2 percent of our ITCR panel 
respondents reported that they were cur- 
rently using BI technologies. Another 21.6 
percent reported that they might use BI 
technologies in some future project. How- 
ever, the majority—39.7 percent—reported 
that they haven’t used BI and have no plans 
to. Another 16.6 percent reported that they 
have no need for BI technologies. 
An Instant Poll, conducted on the SQL 
Server Magazine Web site (http://www 


.sqlmag.com—where youd assume most 
visitors are familiar and knowledgeable 


about SQL Server), showed similar results. 
Thirty-two percent of the respondents 
reported that they use BI regularly and 
another 32 percent reported that they were 
just beginning to get started with BI. How- 
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ever, 36 percent of respondents either said 
they didn’t know what BI was or they didn’t 
even know where to get started using BI. 
Moving to BI is difficult for SQL Server 
organizations steeped in relational technolo- 
gies. Database professionals are often so busy 
dealing with current projects that they aren’t 
ready or able to tackle new technologies. BI 
and data-warehousing technologies are quite 
different from traditional relational database 
technologies. This doesn’t mean that you 
won't use the same skills and tools you use 
as a DBA; you'll definitely build on the skills 
you have. But the database design concepts 
used by BI fact and dimension tables are very 
different from the third normal form that 
relational database designers strive for. The 
MDX language used to query cubes is also 
quite different from good, old T-SQL. These 
basic differences make it difficult for devel- 
opers and DBAs to jump to BI. It might be 
the same database server, but there’s a whole 
new and different set of skills to master. 
Fortunately, there are many resources that 
are designed to help database professionals 
get started with BI. One of the best ways 
to learn about BI is with Microsofts new 
BI videos at http://www.microsoft.com/ 
sql/solutions/bi/videos.mspx and Webcasts 


from the Kimball group at http://www 


-microsoft.com/sql/solutions/bi/kimball 


webcasts.mspx. In addition, in future issues 
of SQL Server Magazine, look for upcoming 


Solutions by Design articles about designing 
for BI and data warehousing by Michelle A. 
Poolet. Michelle will cover a variety of BI 
design topics to help boost you over those 


initial BI barriers. SQL! 
InstantDoc ID 95280 
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Letters 
to the Editor 


Lover Story 


High-End Devices 
and the Real World 
Its great to see articles like 
Kimberly Tripp’s “Essen- 
tial Aspects of Database 
Design” (February 2007, 
InstantDoc ID_94585). I’ve 
added it to the material I 
share with our developers who 
end up designing databases without having 
much background. In the SQL Server world, 
it’s all too common that databases are being 
designed by folks whose main expertise lies 
elsewhere. SQL Server Magazine can do a lot 
to mitigate this problem. 

Regarding fragmentation: Just today I 
was looking at some very high fragmen- 
tation numbers. The situation: The data 
and index files are stored on a dedicated 


In the SQL Server world, 
is too common that 
databases are designed 
by folks whose main 
expertise lies elsewhere. 
SQL Server Magazine 
can do a lot to mitigate 
this problem. 


Equalogics iSCSI disk array. These aren’t just 
iSCSI RAID 10 arrays. One reason for the 
high price is that the device self-balances 
I/O across the spindles as it runs. If it notices 
a trend that spindle 6 is overused and spindle 
3 is underused, it will start migrating blocks 
from 6 to 3 to balance the load. 

So, does fragmentation have any meaning 
on this kind of device? The analysis tools will 
think of that volume as one ordinary disk 
with cylinders and tracks and so forth—but 
it’s a fakeout. SQL Server is writing to the 
device’s interfaces, which take full responsi- 
bility for maintaining the write order and 
the atomicity of the write of blocks that 
are part of a single transaction—even if the 
power goes out. The real blocks may be 
all over the place. Clearly, the one thing it 


won't do is compact down multiple pages 
that can fit into one. With 2TB of disk space 
and 64GB of memory, a cache hit rate of 
99.8 percent, and disk queues under .2/sec 
despite 59 databases used by more than 100 
applications with nearly 3,000 users, I’m 
not sure that compacting it has much point. 
Obviously, Id enjoy fewer articles about 
how to get by with RAID 5 and some more 
real-life stuff with higher-end devices. 
—Roger Reid 


Understanding Subtypes 
I have a question about Michelle A. Poolet’s 
Solutions by Design:“Implementing Super- 
types and Subtypes” (November 2006, 
InstantDoc ID 93241). I enjoyed the article 
and found it to be very helpful. Pm very 
new to data modeling, but I have what 
appears to be an overlapping supertype-sub- 
type structure. I have a supertype of Person 
and subtypes of 
Producer, Agent, 
and Loss_Adjuster. 
A person can be 
any of the three | 
subtypes or any 
two of the sub- 
types together: that 
is, a person could 
be a Producer and 
an Agent or just 
a Producer. My 
question is, can a 
subtype table have 
a relationship with a table outside of the 
supertype-subtype structure? 

—Andy Russell 
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SQL Server Costs in a 
Virtualized Environment 
I read you regularly and have a question 
about SQL Server in a virtualized envi- 
ronment. I pay for SQL Server by the 
processor, but how does this work when 
I deploy SQL Server in a multi-guest 
virtual machine (VM) system in which 
only some of the guests have SQL Server 
installed? For example, I run Windows 
2003 Server and two VMware guests on a 
server with dual processors. Only one of 
the guests runs SQL Server.Am I charged 
twice—once for each processor? 
—Marco Guarnacci 


Microsoft treats VMs as a device. Therefore, 
you would be charged per the number of pro- 
cessors in the VM guest—not the host. If the 
guest is configured with a single processor, you 
would be charged for one processor. 
—Michael Otey 


Absolutely! That’s one of the criteria that I use to 
determine whether I need to specialize: Does one 
subtype have a relationship with other entities in 
the table schema, and does that relationship refer 
only to that subtype? In your case, the subtype 
Producer might have a relationship with an 
entity called Movie, but that relationship wouldn't 
be relevant to the other subtypes. Another 
subtype, Agent, might have a relationship 
with an entity called Studio, and maybe 
even to another subtype of Person not listed, 
called Actor. Subtypes can have relationships 
between themselves, as in Agent to Actor, and 
Actor to Agent. 
I use subtyping a lot when I’m 
modeling conceptually, It really helps me 
to understand the business conditions and 
situations. Whether or not I carry the super- 
type and subtype entities forward, through 
logical modeling into physical modeling, is a 
decision that I'll make on a case-by-case 
basis. SOL 
— Michelle A. Poolet 
InstantDoc ID 95296 
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Tool Time 
with Kevin Kline 


Editor's Note: Post your feedback and tool 
recommendations on the Tool Time forum at 
http://www.sqlmag.com/go/tooltime. 


Analysis Services 
Samples 


Improve efficiency with these tools 


M icrosoft includes with SQL Server 2005 samples that illustrate techniques useful 
for DBAs, developers, and business intelligence (BI) professionals. The SQL Server 
Analysis Services (SSAS) samples are especially useful because they perform many func- 
tions that aren’t otherwise available in SQL Server Management Studio (SSMS) or the 
SQL Server Business Intelligence Development Studio (BIDS). 

In addition, the SQL Server development team made a conscious decision to 
distribute all of the utilities in the samples as source code, not executables. That 
means you'll need to compile the utilities by using either Visual Studio or the .NET 
Framework 2.0 software development kit (SDK) before you'll get a usable binary file. 
Although this requirement might seem like an obstacle, it’s actually a positive feature of 
the samples because you can customize the utilities without going through Microsoft. 

The Microsoft team would love to hear your ideas for improvements to the utili- 
ties. You can submit your comments through the SQL Server Connect page at https:// 
connect.microsoft.com/SQLServer. 


Once you've installed the samples, look in C:\Program Files\Microsoft SQL 


Server\90\Samples\ Analysis Services and its subfolders. You'll find SSAS utilities that 
exploit specific product features. Here are several that you should add to your toolkit. 
(All the utilities listed here except SyncAdvWorksPartitions are in C#.) 

Activity Viewer. This utility shows the current connections, sessions, locks, and traces 
running on any SSAS server that you connect to. It’s similar to the Current Activity 
view in SSMS, except this utility works for SSAS rather than the relational engine. You 
can use Activity Viewer to kill a process. 

AMOBrowser. This utility lets you directly interact with an SSAS server and all 
of its objects by using the Analysis Management Objects (AMO) object model. The 
AMOBrowser works like the SSMS Object Browser; it lets you directly tinker with 
the schema of an SSAS cube and its fact tables, dimensions, and properties. 

AMOAdventureWorks. This tool recreates part of the SSAS AdventureWorks 
database by using AMO, which is nifty because you might want to use AMO to build 
cubes programmatically rather than in the BIDS GUI. Most experienced enterprise 
DBAs strongly encourage you to perform operations like this one in a script as an 
administrative best practice. Scripts provide many benefits over using the GUI, such 
as deeper knowledge of the platform, the ability to run multiple scripts concurrently, 
easier automation, and repeatability. 

BackupAndRestore. This utility describes how to use AMO to back up and restore 
cubes programmatically. You'll find this feature handy for backup automation. 

Display ObjectNames. This tool uses AMO to display object names. I think of this 
one as sort of the “hello world” script for AMO. The purpose of this utlility is to help 
you learn how to use AMO. 

SyncAdvWorksPartitions. This sample SQL Server Integration Services (SSIS) 
package (located in C:\Program Files\Microsoft SQL Server\90\Samples\ 
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Novice 


SQL SERVER ANALYSIS SERVICES 
SAMPLES 


BENEFITS: A great source for techniques 
that aren’t otherwise available in SSMS or 
BIDS 


SYSTEM REQUIREMENTS AND NOTES: 
.NET Framework 2.0; AdventureWorks sample 
database and AdventureWorksDW sample 
cube; and SQL Server 2005 Enterprise, 
Standard, or Developer edition. For more 
detailed notes about how to work with the 
samples, see the forum posting at http:// 
www.sqimag.com/go/tooltime. === 


HOW TO GET IT: Download the SQL Server 
2005 Samples and Sample Databases 
at http://www. microsoft.com/downloads/ 


details.aspx?FamilyID=e719ecf/-9f46-4312- 
af89-6ad8702e4e6e&DisplayLang=en. 


Integration Services\Package Samples\ 
SyncAdvWorksPartitions) synchronizes 
the AdventureWorks database with 
the SSAS cube. You can download 
this sample at http://msdn2.microsoft 
.com/en-us/library/ms161545.aspx. 

Some details about how to compile 
and run the samples are included in two 
SQL Server Samples HTML files that 
ship with the samples. Aside from that, 
the only documentation is in the root 
file of each utility’s Readme file. 

If you have a programming bent, | 
encourage you to develop your own 
variations of the samples and share them 
with the community at the Tool Time 
discussion forum at http://www.sqlmag 
.com/go/tooltime. IF you're not a pro- 
grammer, download the samples, com- 
pile them, and add them to your tool 
kit. They’ll definitely help you be more 
efficient. SOL 

InstantDoc ID 95133 
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Reader Challenge 


2005/2000 
Intermediate 


Change the Status 
of Table items 


T= your SQL Server savvy in this 


solution in an email message to challenge@ 


month’s Reader Challenge. Submit your 


sqlmag.com by April 12. Umachandar Jay- 
achandran, a SQL Server Magazine technical 
editor, will evaluate the responses. We’ll 
announce the winner in an upcoming 
SQL Server Magazine UPDATE. (Make 
sure you're signed up for this free weekly 
email newsletter at http://www.sqlmag 
.com/email.) The first-place winner will 
receive $100, and the second-place winner 


will receive $50. 


è 
Problem: INSERT INTO 
iva i t veloper for INSERT INTO 
Siva is a database developer fo INSERTE NIO 
a company that sells products INSERT INTO 
l . INSERT INTO 
online. The company stores its 
; -INSERT INTO 
product inventory and order- (eyes NC 
processing data in a SQL Server INSERT INTO 
INSERT INTO 
2000 database. For the purpose INSERT INTO 
of the problem, use the tables INSERT INTO 
P 9 INSERT INTO 
in Listing 1 for the order-pro- INSERT INTO 
sung p INSERT INTO 
cessing data. INSERT INTO 
INSERT INTO 


The Orders table contains 
the details of each customer’ online order 
form. The OrderDetails table contains the 
line items for each order. The status of the 
order is tracked at both the order and order 
detail levels, although the status values are 
different from table to table. Listing 2 shows 
some sample data for the tables. 

Orders are created with at least one 
line item in a single transaction. The status 
change for a particular order happens in 
the following sequence: Orders are initially 
created with a “Processing” status. Once all 
of the line items in an order have been pro- 
cessed, the status changes to “Shipped.” If the 
order is cancelled, the status is set to “Can- 
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PP USTING I Code for Creating the Orders and OrderDetails Tables 


CREATE TABLE Orders 
¢ 


OrderId int NOT NULL PRIMARY KEY, 
CustomerId int NOT NULL, 
OrderDate smalldatetime NOT NULL, 
OrderStatus tinyint NOT NULL 
CHECK (OrderStatus IN (1 /*Processi 
) 


CREATE TABLE OrderDetails 
¢ 


PRIMARY KEY (OrderId, ProductId), 
OrderId int NOT NULL FOREIGN KEY REFERE 
ProductId int NOT NULL, 
Quantity int NOT NULL CHECK(Quantity > 
OrderDetailStatus tinyint NOT NULL 
CHECK( OrderDetailStatus IN (1 /*Pr 
/*Cancelled*/, 4 /*Returned*/)), 


Orders 
Orders 
Orders 
Orders 
Orders 


(OrderiId, 
(OrderiId, 
(Orderid, 
(OrderiId, 
(OrderiId, 


Customerld, 
Customerld, 
Customerld, 
Customerld, 
Customerld, 


OrderDate, Ord 
OrderDate, Ord 
OrderDate, Ord 
OrderDate, Ord 
OrderDate, Ord 


OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 
OrderDetails 


(Orderld, 
(Orderld, 
(Orderid, 
(Orderld, 
(Orderld, 
(Orderlid, 
(Orderld, 
(Orderld, 
(Orderld, 
(Orderid, 
(Orderld, 


celled.” A batch processes the OrderDetails 
entries, then updates the status of each 
line item. At the end of the batch process, 
Siva wants to update the Orders table and 
mark all orders that have been completely 
processed. 


ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 
ProductId, Quantity, 


Help Siva complete the following tasks: 


1.Write an UPDATE statement that 
can change the order status according 
to the status of the line items. 

2. Optimize the schema by adding any 
additional indexes to help with the 
processing of the UPDATE statement 
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ng*/, 2 /*Shipped*/, 3 /*Cancelled*/) ) 


NCES Orders(OrderId), 
0), 


ocessing*/, 2 /*Processed*/, 3 


PP LISTING 2 Sample Data for the Orders and OrderDetails Tables 


erStatus) VALUES(1, 1, '20070121', 2) 
erStatus) VALUES(2, 1, '20070201', 3) 
erStatus) VALUES(3, 1, '20070201', 1) 
erStatus) VALUES(4, 2, '20070101', 2) 
erStatus) VALUES(5, 3, '20070202', 1) 
OrderDetailStatus) VALUES(1, 1, 1, 2) 
OrderDetailStatus) VALUES(1, 2, 2, 2) 
OrderDetailStatus) VALUES(2, 3, 4, 3) 
OrderDetailStatus) VALUES(3, 10, 2, 1) 
OrderDetailStatus) VALUES(3, 11, 1, 2) 
OrderDetailStatus) VALUES(3, 21, 3, 2) 
OrderDetailStatus) VALUES(4, 1, 2, 2) 
OrderDetailStatus) VALUES(4, 11, 2, 4) 
OrderDetailStatus) VALUES(4, 3, 5, 2) 
OrderDetailStatus) VALUES(4, 33, 1, 2) 
OrderDetailStatus) VALUES(5, 5, 1, 1) 


or queries that look at orders of a 
particular status. SOL 


InstantDoc ID 95324 


e on the WEB 


Download the listings at 
InstantDoc ID 95324 


Umachandar Jayachandran 
(umachandar@yahoo.com) works in the SQL Performance Engineering 
Team at Microsoft. He has over 12 years of experience working with SQL 
Server in OLTP and data warehouse environments. He is also a contrib- 
uting author of SQL Unleashed, 2nd Edition (SAMS). You can 

find his contributions at http://blogs.msdn.com/sqltips and http:// 


blogs.msdn.com/sqlperf. 


www.sqimag.com 


Reader to Reader 


A Quick and Easy Way to View Errorlog Files 


I created a stored procedure, sp_ShowErrors, to view SQL 
Server errorlog files. In its default configuration, this stored 
procedure takes the two most recent errorlog files and pro- 
duces a report that highlights errorlog entries of interest. To 
give context to those errorlog entries, the report also includes 
(but doesn’t highlight) the preceding and following four 
entries in the errorlog file. 


> LISTING | Excerpt from the sp_ShowErrors Stored Procedure 


WHILE (acount < dlogs) 
BEGIN 
IF (@count = Ø) 
BEGIN 
INSERT INTO #errorlog values 
INSERT INTO #errorlog values ('',' ',@) 
INSERT INTO #errorlog values ('','*žžkkkkkkkkkkkkkk! 1) 


CONT 
ee 
vA 

INSERT INTO #errorlog values ('','* LOG: CURRENT *',@) 
vi 
z 
va 


',0) 
This stored procedure has saved me countless hours of INSERT INTO #errorlog values ('',!#k#eRERRREEREEE ID) 

i rth rs. For ex l i howEr- INSERT INTO #errorlog values ('' 
tedium over the years. For example, by adding sp_Show TCER Ne GNVUT He ceotre ae ice eae 
rors to a batch process that runs on more than 40 servers and END 


f : ELSE 
concatenating the results to one file, I can review the errorlog BEGIN 


5 5 2 a INSERT INTO #errorlog values ('',' ',Q) 
files for all the servers in a matter of minutes. It’s a great alter ANCESTOR CEEOL (cree eces (000 9) 
native to logging on to each server and manually reviewing INSERT INTO #errorlog values ('',' ',Ø) 
es INSERT INTO #errorlog values ('',' ',@) 

the errorlog files through the GUI or not even reviewing INSERT INTO #errorlog values (1! '#kekRR EKER EERE! ()) 
SELECT doutput = '* LOG: CURRENT - ' + 


them at all due to time constraints. I have performance-tuned 

sp_ShowErrors so that it can handle even the largest errorlog 

file that would cause Enterprise Manager to choke. INSERT INTO #errorlog values ('',' ',) 
Listing 1 shows an excerpt from sp_ShowErrors. (You can Abe A ee ee > 

download the entire stored procedure from the SQL Server 

Web site.) As callout A shows, sp_ShowErrors uses the 

sp_readerrorlog stored procedure. Note that sp_readerrorlog 

is undocumented, so you need to use at your own risk. RT Eaa Se 
I wrote sp_ShowErrors for SQL Server 2000. To get it END 

working in SQL Server 2005, you'll need to modify the code 

because the output of sp_readerrorlog in SQL Server 2005 is 

different. I hope you find this stored procedure as useful as I do. 


convert(varchar(3),@count) + ' *' 
INSERT INTO #errorlog values ('',@output,0) 
INSERT INTO derrorlog values ('','*žžkkkkkkkkkkkkkkkkkk! | 0) 


(A) SELECT ASAL = 'exec master..sp_readerrorlog ' + 
convert (varchar(3),@count) 


INSERT INTO #errorlog (log_entry, continuationrow) 


—Bill McEvoy 
InstantDoc ID 95134 


UDF Eliminates 

Concatenation Woes 

You probably know how to concatenate 
multiple values from multiple rows of a 
table. This technique is useful when you 
need to provide multiple values in a single 
variable. For example, when you're calling 
a stored procedure, you might need to pass 
in multiple values in a single parameter, as 
Listing 2 shows. 

However, when you use this type of 
concatenation, you're limited to 8,000 char- 
acters, which can be problematic in several 
ways. First, you might need many local 

more anite WEB variables to hold all 
— the values in a table. 
instant Ds geia Y Then, there's the 
95135, anq 9187 problem of trying 

to figure out which variables are full and 
which variables still have room for more 
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values. Another problem can occur when a 
table value is too large for a local variable. 
In such cases, you might need to split a 
table value across multiple local variables. 
You might even be forced into some kind 
of cursor or looping structure to address this 
particular problem. 

If you’ve experienced any of these con- 
catenation woes, have I got the solution for 
you: dbo.fn_AddToBuffer. Although dbo 
fn_AddToBuffer won't eliminate unwanted 
belly fat or make you rich quick, it will 
eliminate concatenation problems and make 
your job easier because it uses any number 
of local variables as a buffer to work around 
the 8,000-character limitation. As Listing 
3, page 10 shows, using this user-defined 
function (UDF) requires a bit more typing, 
but you end up with an elegant solution 
that’s sumple to read and works well. This 
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UDF works on SQL Server 2005 and SQL 
Server 2000. 

Here’s how you use dbo.fn_AddTo- 
Buffer. The UDF needs three parameters. 
The first parameter is a varchar(8000) 
variable that represents the value you want 
to add to the “buffer” (i.e., the set of local 
variables). The second parameter is an 


LISTING 2 Using Concatenation 
to Provide Multiple Values in a Single 
Parameter 


DECLARE ax varchar (8000) 

SELECT 9x = 1 

SELECT ax = ax + InvoiceNumber + ' ' 
FROM dbo.InvoiceTable 

EXEC dbo.SomeProcedure @x 


Share Your Experiences 

Share your SQL Server code, comments, discoveries, and solutions to 
problems. Email your contributions to r2r@sqlmag.com. Please include 
your full name and phone number. We edit submissions for style, 
grammar, and length. If we print your submission, you'll get $100. 
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integer that represents the starting buffer 
position for each local variable. The third 
parameter is an integer that represents the 
total buffer length prior to adding the cur- 
rent value. The UDF will evaluate just how 
many characters and which portion of the 
input value should be parsed and returned 
for each local bufter variable. 

You can use dbo.fn_AddToBufter for 
any number of local variables. In each local 
variable, it uses every available character 
position. I’ve found this function most 
useful when the called stored procedure 
performs dynamic SQL execution. When 
you use the EXEC statement for dynamic 
SQL, there’s no practical limitation to how 
long the statement can be, so concatenating 
multiple 8,000-character strings together 
isn’t a problem. 

For example, suppose you have a 
SELECT statement that includes an IN 
clause. You can provide the IN values as 
parameters so that the dynamic SQL state- 
ment looks something like 


EXEC ('SELECT * FROM 
InvoiceDetails WHERE 
InvoiceNumber IN (' 
+ 0x2 + @x3 + ax4 
+ @x5 + ')' " 


+ ax1 


In the example in Listing 3, the values 
need to be delimited by a space, so I added 
a single space to the end of each Invoice- 
Number. However, in other situations, you 
might not need a delimiter or you might 
need a different delimiter. Your values might 
not even be characters (e.g., they're INT 
values), or you might need your values 
properly quoted. You can determine what 
you need in the SELECT LIST of the 
subquery by using the CAST or CON- 
VERT operators and adding your preferred 
delimiter and/or quotation marks. 

If you often need to provide multiple 
values in a variable but find a single vari- 
able too limiting, give dbo.fn_AddToBuffer 
a try. This UDF isn’t available at any store, 
and you won't find it advertised on TV. You 
can only find it on SQL Server Magazine’s 
Web site. 

—Lawrence Rogers 
InstantDoc ID 95135 
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Procedure Helps Pinpoint 
Problematic Processes 

If your SQL Server 2005 machine is expe- 
riencing performance problems and you 
need to quickly see all the T-SQL code 
currently executing on it, you'll likely be 
interested in sp_display_curr_tsql_in_ses- 
sions. This stored procedure displays all 
the T-SQL statements currently being 
executed along with the IDs of the pro- 
cesses running those statements. 

As Listing 4 shows, sp_display_curr_ 
tsql_in_sessions loops through all the 
current sessions it finds in the sys. 
sysprocesses system view. For each ses- 
sion, the stored procedure fetches the 
sql handle. It uses this handle with the 
fn_get_sql system function to obtain the 
T-SQL statement that the session is cur- 
rently running. The fn_get_sq] function is 
similar to the DBCC INPUTBUFFER 
statement in that both return the T-SQL 
statement that the specified session is 
currently executing. However, I prefer 


LISTING 4 The sp_display_curr_tsql_ 
in_sessions Stored Procedure 


USE master 
GO 
CREATE PROCEDURE sp_display_curr_tsql_ 
in_sessions 
AS 
BEGIN 
SET NOCOUNT ON 
DECLARE @currSID int 
DECLARE atsqlText varchar (max) 
DECLARE @sqlHndl varbinary(20) 
DECLARE c_spids CURSOR FOR SELECT 
spid 
FROM sys.sysprocesses 
CREATE TABLE #currS@QLinfo (spid 
int, sqltext text) 
OPEN c_spids 
FETCH c_spids into acurrSID 
WHILE @afetch_status = 0 
BEGIN 
SELECT asqlHndl = sql_handle 
FROM sys.sysprocesses 
WHERE spid = dcurrSID 
SELECT atsqlText = convert 
(varchar (max),Ctext J) 
FROM ::fn_get_sql (a@sqlHndl) 
IF ISNULL (@tsqlText,'*') != 
Iki 
BEGIN 
INSERT #currSQLinfo VALUES (3 
currSID,atsqlText) 
END 
FETCH c_spids INTO adcurrSID 
END 
CLOSE c_spids 
DEALLOCATE c_spids 
SELECT * FROM #currS@QLinfo 
DROP TABLE #currS@Linfo 
SET NOCOUNT OFF 
END 
GO 


using fn_get_sql because it displays all the text in the T-SQL statement, whereas DBCC 
INPUTBUFFER returns only the first 255 characters in the T-SQL statement. The 
sp_display_curr_tsql_in_sessions stored procedure stores the session IDs and T-SQL 
statements in a temporary table, which it later removes. 

To execute the sp_display_curr_tsql_in_sessions stored procedure, you use the fol- 


lowing statement: 


exec sp_display_curr_tsql_in_sessions 


As you can see, it doesn’t need any parameters. 

I wrote sp_display_curr_tsql_in_sessions for use on SQL Server 2005. With a modi- 
fication, you can get it to work on SQL Server 2000 Service Pack 3 (SP3) and later. 
(The stored procedure won't work on previous versions of SQL Server 2000 because 
the fn_get_sql function was introduced in SP3.) You'd need to use varchar(8000) instead 
of varchar(max) with the CONVERT function because varchar(max) is new to SQL 


Server 2005. 


[SQL] 
—Eli Leiba 
InstantDoc ID 95187 


LISTING 3 Using dbo.fn_AddToBuffer to Provide Multiple Values 
in a Few Parameters 


DECLARE @x1 varchar(8000), ax2 varchar(8000), x3 varchar(8000), 
ax4 varchar(8000), @x5 varchar(8000), @RunningLength int 


SEuECy aki] = 7°. 
SELECT @x1 = 
, 3x2 = 
pn WES 
7 3x4 
, 3x5 = 
, @RunningLength = 
FROM (select InvoiceNumber + ' 


ax2 = '', ax3 = '', 


Ly E 


ax4 
ax1 + dbo.fn_AddToBuffer(InvoiceNumber, Ø, @RunningLength) 
ax2 + dbo.fn_AddToBuffer(InvoiceNumber, 8000, @RunningLength) 
ax3 + dbo.fn_AddToBuffer(InvoiceNumber, 16000, @RunningLength) 
ax4 + dbo.fn_AddToBuffer(InvoiceNumber, 24000, @RunningLength) 
ax5 + dbo.fn_AddToBuffer(InvoiceNumber, 32000, @RunningLength) 
@RunningLength + LEN(InvoiceNumber) 

' AS InvoiceNumber FROM dbo. 


= 0) ax5 , @RunningLength = Ø 


InvoiceTable) AS Z 


EXEC dbo.SomeProcedure @x1, ax2, @x3, @x4, @x5 
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2005 
Intermediate A 


Layer on the protective 
measures before, during, and 
after SQL Server installation 


icrosoft’s SQL Server 2005 is an enterprise-class database server, 

WB suitable for the toughest workloads and boasting some impressive 

security features. Yet enterprises might be placing themselves at risk if 

they simply deploy SQL Server and neglect to take advantage of the 
security features. 

In general, Microsoft recommends a defense-in-depth approach to 
securing infrastructure components such as database servers. This approach 
focuses on perimeter, network, host, application, data, and physical defenses. 
Here are 10 steps to securing SQL Server 2005 that you should take before, 
during, and after installation, based on the defense-in-depth approach. 


| i 2 Unnecessary Services 

Properly preparing for an installation of SQL Server 2005 is one of 
the most important ways you can protect your databases and the data 
contained within them. 
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The first step is to carefully prepare the server on which you'll 
install SQL Server 2005. In particular, the server should be dedi- 
cated to running SQL Server 2005. It should not also be used to 
run other services such as Microsoft IIS, provide general Web-server 
functionality, serve applications, or be configured as a domain con- 
troller (DC). Its important that a server dedicated to SQL Server 
2005 not run other services because if those other services become 
compromused, perhaps due to poor configuration, an attacker might 
be able to gain system-level access to the host OS and then to SQL 
Server 2005 and the databases it manages. 

All unnecessary services and applications should be removed. For 
example, if youre repurposing an existing server that was a DHCP 
or DNS server, you shouldn’t just disable those services, you should 
remove them by using the Optional Components Wizard that’s 
part of Add/Remove Programs in Control Panel. Likewise, remove 
all third-party applications that aren’t required, such as WinZip 
and Microsoft Office applications. You should also disable file and 
printer sharing. If you don't need them, I recommend that you also 
disable the administrative shares (e.g., C$, D$, ADMINS). 

Note that depending on what features of SQL Server 2005 you 
wish to install (e.g., Reporting Services), you might need to install 
IIS and .NET Framework 2.0. If this is the case, you should limit 
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the use of these additional services and features to SQL Server 2005 
only, and you should configure them as securely as possible by using 
the appropriate best practices for each. Security best practices and 
guidance for Microsoft technologies can be found at http://www 
-microsoft.com/technet/security/guidance/default.mspx. You 
might also want to check http://msdn.microsoft.com/security. 


Apply All Service Packs and Security 

Updates to the Host OS 
The second step in preparing for installation of SQL Server 2005 
also concentrates on the host and is to ensure that all the latest 
service packs and software updates are applied to the OS on the 
server on which you'll install SQL Server 2005. This step is crucial 
to securing your SQL Server 2005 installation. Just as attackers can 
gain access to your system through services and applications run- 
ning on it, they can get access to the host through vulnerabilities 
in the OS itself. 

Visit _http://update.microsoft.com to ensure that you have the 
latest service packs and updates, or use the Windows Update or 
Microsoft Update tool on the Start menu if you don’t have an 
enterprise software update distribution mechanism such as Micro- 
soft Systems Management Server (SMS). 


by John Howie 
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Limit Access 

to the Host 
The third step is a combination of focus on the 
host, perimeter, and physical environment and 
is to restrict access to the server. An attacker 
has several ways to gain access to a server, 
including interactively, over the network, and 
physically when the OS isn’t running. 

You limit interactive access to the host by 
ensuring that the local SAM database con- 
tains no unnecessary accounts and that only 
permitted domain accounts can be used to 
log on to the system. You can examine and 
remove accounts in the local SAM data- 
base by using the Microsoft Management 
Console (MMC) Computer Management 
snap-in (compmgmt.msc). 

To prevent domain accounts from log- 
ging on interactively to the server and 
accessing a desktop, you can use Group 


Security 


and Deny log on through Terminal Services settings 
to configure who can log on interactively. 

You'll need to pay specific atten- 
tion to members of the Domain Admins 
group—those users who have administra- 
tive access to all systems (DCs, worksta- 
tions, and servers) in a domain. In many 
cases, Domain Admins members will also 
be database administrators, but in some 
environments, they won't. You might want 
to restrict access to database servers for 
members of Domain Admins. You can do 
this most simply by removing Domain 
Admins from the local Administrators 
group and adding the account names of 
trusted administrators to this group. 

To prevent remote attackers from 
accessing the server over the network, you 
should place the server in a secured net- 
work segment and use at least one firewall 
(including the host 
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firewall that comes 
with Windows 
Server 2003 Ser- 
vice Pack 1—SP1) 
or routing rules to 
deny access to users 
who dont need to 
access the server. For 
users who are per- 
mitted access to SQL 
Server 2005, you 
should deny access 
to any services other 
than SQL Server 
2005 that the users 


don’t need to access. 


> > FIGURE | Configuring services and connections Although you hard- 


Policy or the MMC Local Security Settings 
snap-in (secpol.msc) to set User Rights 
Assignment. If using Group Policy, expand 
Computer Configuration, Windows Set- 
tings, Security Settings, Local Policies, and 
User Rights Assignment. If using the Local 
Security Settings snap-in, expand Local 
Policies, User Rights Assignment. 

Use the Allow log on locally and Deny logon 
locally settings to configure who is permitted 
to log on at the console. If you permit access 
to your server through Termunal Services, even 
if only for remote administration, you should 
also use the Allow log on through Terminal Services 
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ened the host in step 
1, Windows will always have some services 
running which listen to the network for 
inbound traffic. 

Last in this step, ensure that the server 
is physically secured. If attackers can gain 
physical access to the server, they can boot 
an alternative OS and reset the local Admin- 
istrator password to log on or they can gain 
direct access to files on the disks, including 
database files. Note that if you're running 
SQL Server 2005 on a laptop, you might 
never be able to completely address the risk 
of physical access to the host. In such cases, 
step 10, below, can help. 
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Create Accounts for 

SQL Server to Run Under 
The last step in preparing for a secure installa- 
tion of SQL Server 2005 is to create accounts 
under which SQL Server 2005 services will 
run. The number of accounts you'll need 
depends on two factors: the number of SQL 
Server 2005 features you intend to install and 
the number of SQL Server 2005 installations 
in your environment. 

Although you can use one account for 
all SQL Server 2005 services, I recom- 
mend that you create an account for each. 
You'll need at least three accounts: one 
for the SQL Server database engine, one 
for the SQL Server Agent, and one for 
the SQL Server Browser. You'll also need 
service accounts for Analysis Services and 
Reporting Services, if you intend to install 
these components. 

I also recommend that you use unique 
service accounts for each installation of SQL 
Server 2005 in your environment—that is, 
create a unique account for each service on 
each installation. (An exception would be 
using one database engine service account 
for a cluster of servers or for servers that 
replicate data between themselves.) 

If you're installing SQL Server 2005 
in an environment in which the database 
engine will never need access to other 
servers and services across a network, you 
can use local accounts instead of domain 
accounts. Regardless of whether you use 
a domain account or a local account, the 
account shouldnt have privileges beyond 
those of an ordinary user—that is, it should 
be a member of Domain Users or Users 
only. Make sure that the passwords on the 
accounts you create are strong, and set up a 
schedule to change them on a regular basis. 


5 Install SQL Server 2005 


Once the server has been prepared, you can 
begin installing SQL Server 2005. Step 5 
in securing SQL Server 2005, the careful 
selection of installation options, addresses 
the application aspect of the defense-in- 
depth strategy. 

The first selection you need to make is 
which components to install. You should 
not install components of SQL Server 2005 
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that you don’t need. (Remember, the goal 
is to minimize the attack surface.) I recom- 
mend that when prompted for Components 
to Install in the Setup wizard, you click 
Advanced to select not only top-level fea- 
tures but also the individual features of each 
top-level feature. For example, the Advanced 
option lets you decide whether to install 
replication or full-text search features along 


The SQL Server Surface 
Area Configuration tool 
is the preferred means 
to make securily- 
related adjustments to 
your SQL Server 2005 
configuration. 


with the default Database Services. If you're 
installing a single instance of SQL Server 
2005 that won't replicate data to other 
database servers and if you don’t intend to 
use fast text searching, you can simply omit 
these features from the installation. 

Enter carefully the credentials of each 
account you created in step 4 for each of 
the services when prompted. When asked 
to select an authentication mode, choose 
Windows Authentication Mode wherever 
possible. In some situations, you'll have 
to use Mixed Mode, which permits both 
Windows Authentication and SQL Server 
Authentication. If you select Mixed Mode, 
you must enter a password for the SQL 
Server systems administrator (sa) account. 
As with the service accounts you created in 
step 4, you should ensure that the password 
is strong and set up a schedule to change it 
on a regular basis. 


After SQL Server 
Installation: Apply SQL 
Server 2005 Service 
Packs and Updates 
The first post-installation step, which also 
addresses the application aspect of defense 
in depth, is the application of all SQL Server 
2005 service packs and software updates. 
At the time of this writing, Microsoft has 
released a Community Technology Preview 
(CTP) of SQL Server 2005 SP2. 


Run the SQL Server 
Surface Area Configuration 
Tool 
The next step, another application-level one, 
is to run the SQL Server Surface Area Con- 
figuration tool. Although you can launch 
the tool immediately after you install SQL 
Server 2005, I recommend that you run it 
after applying service packs and updates. I 
also recommend that you run it after you 
make changes to the configuration of SQL 
Server 2005 or add or remove components. 
You can find the SQL Server Surface 
Area Configuration tool in the Configura- 
tion Tools submenu under Microsoft SQL 
Server 2005 on the 
Start menu. When 
you launch the tool, 
it’s pointed at loc- 
alhost, but you can 
also use the tool to 3 WSSCLSERVER 
configure remote i 
installations of SQL 
Server 2005. In the 
tool, you can choose 
either Surface Area 
Configuration for Ser- 
vices and Connections 
or Surface Area Con- 
figuration for Features. 
The former allows 
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anyone with a network monitor can eaves- 
drop and gain access to potentially sensitive 
information. SQL Server 2005 supports the 
use of Secure Sockets Layer (SSL) to both 
encrypt traffic between the client and the 
server and guarantee the identity of the 
server, which helps mitigate the risk of man- 
in-the-middle attacks. 

To secure communications with SSL, 
you must obtain a certificate from an 
enterprise public key infrastructure (PKI) 
built with Microsoft’s Certificate Services 
or from a third-party vendor such as Veri- 
Sign. The SSL certificate should contain 
the host name or Fully Qualified Domain 
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you to view the 
installed SQL Server 
2005 services and 
their configuration 
either by instance or by component (see 
Figure 1). The latter allows you to view 
specific features that have security implica- 
tions and enable or disable each (see Figure 
2). The SQL Server Surface Area Configu- 
ration tool is the preferred means to make 
security-related adjustments to your SQL 
Server 2005 configuration. 


Encrypt Traffic Between 

the Database Server and 

Database Clients 
To address the network aspect of the 
defense-in-depth model, you must secure 
communications between the database 
server and applications. By default, com- 
munications between a database client and 
the database server are sent in clear text, and 
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> > FIGURE 2 Configuring features 


Name (FQDN) of the database server or 
cluster as appropriate and should be in the 
local computer certificate store if SQL 
Server 2005 is running in the LocalSystem 
context; otherwise, the certificate should 
be in the certificate store for the account 
under which the SQL Server database 
service runs. 

Once the certificate is installed, launch 
SQL Server Configuration Manager, 
expand the SQL Server 2005 Network 
Configuration node, right-click the Pro- 
tocols subnode, and select Properties. On 
the Certificate tab, select the correct cer- 
tificate from the Certificate drop-down list 
(see Figure 3, page 16). On the Flags tab, 
under General, select Force Encryption 
and change the value to Yes by selecting 
it from the drop-down list. Once you’ve 
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configured SQL Server 2005 to use the 
certificate, you need to stop and restart 
SQL Server 2005 for the changes to take 
effect. 

A second means for encrypting traffic 
between a database client and the database 
server is IPsec. The benefit of IPsec is that 
it doesn’t require any configuration of SQL 


to allow system administrators access to the 
database server and databases in this way. 
You might instead want to grant named 
user accounts sysadmin role membership 
and deny Administrators and SYSTEM the 
sysadmin role. This is especially true if you're 
running multiple instances of SQL Server 
2005 on one system, such as in a hosting 
environment. 
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By extension, when 
planning which databases 
will be hosted on a SQL 
Server 2005 instance, you 
should ensure that only 
the minimum permis- 
sions required be granted 
to users to access the 
databases for SELECT, 
INSERT, UPDATE, and 
DELETE operations, 
and to execute stored 


Ua 


procedures. SQL Server 
2005 supports a more 


FIGURE 3 Configuring secure communications with SSL 


Server 2005, and it can be used to secure 
all traffic (not just SQL Server 2005 traffic) 
and to restrict access to the database server 
to only other domain-joined systems. For 
more details about isolating your SQL 
Server 2005 system by using IPsec, visit 
http://www.microsoft.com/sdisolation. 


Configure Login Privileges 

and Server Roles 
The next step in securing SQL Server 
2005 is to configure login rights and server 
roles. For example, by default, members of 
the local Administrators group are granted 
login privileges. They're also granted mem- 
bership in the sysadmin role, which gives 
them nearly unrestricted access to both the 
database server and the databases it man- 
ages. Members of the SYSTEM group, 
which represents the OS, also have sysadmin 
membership. 

In most environments this access isn’t an 

issue, but in some, it might not be desirable 
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fine-grained approach 
to security than SQL 
Server 2000, so you can 
apply more granular 
rights, including to users 
who perform mainte- 
nance functions. 


1 Encrypt the 

Databases 
The last step in securing your SQL Server 
2005 system is protecting the data within 
your database files when SQL Server 
2005 and the host OS aren’t running. This 
step, which addresses the data aspect of 
the defense-in-depth model, encrypts the 
databases. 

There are a number of approaches to 
encryption in SQL Server 2005, including 
support for encryption within the databases 
themselves. The simplest approach, however, 
where the main concern is one of stolen 
disks, backup tapes, or laptops, is to use Win- 
dows’ Encrypting File System (EFS). SQL 
Server 2005 databases can be encrypted by 
using EFS and still be useable by the data- 
base server. 

To encrypt databases with EFS, you must 
log on interactively to the database server by 
using the credentials that SQL Server runs 
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under and encrypt the database and transac- 
tion log files by using Windows Explorer or 
the cipher.exe command-line utility. This 
means that SQL Server must run under a 
named user account, not under the Local- 
System account or one of its derivatives 
(LocalService or NetworkService). If you 
do choose to use EFS to encrypt database 
and transaction log files, I recommend that 
you ensure that a named Data Recovery 
Agent is configured so that the databases 
can be recovered if, for whatever reason, the 
service account is locked out or deleted. 
Note that using EFS to encrypt a SQL 
Server database will slow performance 
somewhat, but not as much as you might 
expect. On a suitably equipped server, you 
shouldnt encounter problems unless the 
database is heavily used. The performance 
hit is usually negligible on desktop and 
laptop systems, and the security that comes 
with EFS would easily outweigh any slight 
slowdown, especially on laptops containing 
customers’ personal information. 


SQL Server 2005 Security 
in Depth 
SQL Server 2005 contains great security 
features and tools, but enterprises can still 
be at risk if you don’t install and configuring 
the database server correctly. The 10 steps I 
outlined here are practical, easy to follow, 
and will significantly reduce exposure to risk 
by addressing key areas of concern when 
installing and running SQL Server 2005. 
You can read more about the defense- 
in-depth approach on which I’ve based 


this article at http://www.microsoft.com/ 


technet/security/bestprac/overview.mspx. 


This overview also contains a link to Im- 
proving Web Application Security, “Chapter 
18: Securing Your Database Server” which 
discusses securing SQL Server 2000. Much 
of the advice in the chapter is valid for all 
versions of SQL Server and is good back- 
ground reading for SQL Server 2005 users, 
but the chapter doesn’t include advice spe- 
cific to SQL Server 2005. SQL] 
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Pei Feature 


by Yoel Martinez 


a 


Compressing and decompressing their data i 
» just got a whole lot easier l 


A 
anipulating binary large objects (BLOBs) and character large objects (CLOBs) has always been f 
difficult in SQL Server. Fortunately, SQL Server 2005 provides three new data types for large 
object (LOB) storage. With these new data types, you can easily manipulate BLOBs and CLOBs, 
especially when you use Common Language Runtime (CLR) procedures and functions. With the / 
compression algorithms in Microsoft .NET Framework 2.0, you can create CLR functions that seam- j 
lessly compress and decompress LOBs with minimal performance impact. | D> - on the WEB 
= Before I show you how to do so, let me introduce you to the three new Download the listings at 
; InstantDoc ID 95185. 


data types that make this compression and decompression possible. 
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BLOBs and CLOBs 


The New Kids on the Block 

SQL Server 2005 provides three new data 

types to store and manipulate LOBs: 

e varbinary(max), which you use for 
BLOBs 

evarchar(max), which you use for 
CLOBs 

envarchar(max), which you use for 
CLOBS with Unicode values 


The new varbinary(max), nvarchar(max), and 
varchar(max) data types behave much like 
the traditional varbinary(n), nvarchar(n), and 
varchar(n) data types, respectively. However, 
the maximum storage size of the three new 
data types is 231-1 bytes, or about 2GB. 

The new data types replace the text and 
image data types from previous versions of 
SQL Server. SQL Server 2005 Books Online 
(BOL) states that you shouldn’t use the text 
and image data types in new SQL Server 
2005 applications and you should change 
any existing text and image data types to the 
new data types in legacy applications. 

Unlike text and image values, 
varbinary(max), nvarchar(max), and 
varchar(max) values can be used for variables 
and function parameters. They can also be 
returned by CLR (or T-SQL) scalar-value 
functions. These traits make them great 
candidates for data manipulation. 

CLR procedures and functions use 
the .NET data types of SqlBytes and Sql- 
Chars to receive and process BLOBs and 
CLOBs, respectively. SqlBytes is used to 
pass varbinary(max) parameters, where as 
SqlChars is used to pass nvarchar(max) and 
varchar(max) parameters. CLR is Unicode 
based, so passing varchar(max) as a parameter 
implies converting it to Unicode before 
parameters are passed. SQL Server 2005 
does all this without your intervention, 
but it’s important that you're aware of the 
differences concerning Unicode and non- 
Unicode strings. 


Compressing LOBs 

Before SQL Server 2005, you had to use 
extended stored procedures to compress 
data, which is a difficult and risky business. 
SQL Server 2005s CLR integration capa- 
bilities make compression less complicated, 
more secure, and more stable. 
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p> LISTING I The Compression Function 


using 
using 
using 
using 
using 
using 
using 


System; 

System.Data; 
System.Data.SqlClient; 
System.Data.SqlTypes; 
Microsoft.SqlServer.Server; 
System.10; 
System.10.Compression; 


public partial class UserDefinedFunctions 
Ht 


// Set the function characteristics. 


[CMicrosoft.SqlServer.Server.SqlFunction(IsDeterministic=true, 


DataAccess=DataAccessKind.None) J 


@ public static SqlBytes fn_compress(SqlBytes blob) 
{ 
if (blob.IsNull) 
return blob; 
// Retrieve the BLOB's data. 
byteL] blobData = blob.Buffer; 
© // Prepare for compression. 


MemoryStream compressedData = 
DeflateStream compressor = 
CompressionMode.Compress, true); 

// Write the uncompressed data using 


new MemoryStream(); 
new DeflateStream(compressedData, 


a DeflateStream compressor. 


compressor.Write(blobData, Ø, blobData.Length); 


// Close the compressor to allow all 
compressor.Flush(); 
compressor.Close(); 
compressor = null; 


// Return the compressed blob. 
return new SqlBytes(compressedData); 


3; 


Because you can convert data back and 
forth between BLOBs and CLOBs, lets 
explore how to use varbinary(max) for simpli- 
city sake. Listing 1 contains a function, written 
in C#, to compress BLOBs. Compressing a 
BLOB is as easy as passing in a SQLBytes 
parameter, reading the BLOB’ data, and 
writing that data to a compression stream. 

Callout A in Listing 1 shows where 
the compression function receives a 
varbinary(max) value as a SQLBytes param- 
eter. SQLBytes represents a mutable type 
that wraps either an array or a stream. 
Assuming it wraps an array, the code uses 
the Buffer property to access the under- 
lying array and retrieve the BLOB’s data, as 
callout B shows. Note that for large BLOBs, 
this method could raise an out-of-memory 
exception if youre running SQL Server 
2005 Service Pack 1 (SP1) or earlier. For- 
tunately, unlike extended-procedure excep- 
tions, CLR exceptions don’t crash SQL 
Server 2005 machines. This size limitation 
has been resolved in SQL Server 2005 
SP2. 

Finally, the compression function uses the 
DeflateStream class in .NET Framework 2.0 
to compress the BLOB, as callout C shows. 
The compression stream writes toa Memory- 
Stream object that’s later used to create a 
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the compressed bytes to be written. 


new SQLBytes object that’s returned to 
the calling process. There 1s only one caveat: 
Microsoft’s implementation of Deflate- 
Stream requires the stream to be closed 
before it writes the last compressed bytes. 
Flushing alone isn’t enough. So, the function 
flushes, then closes the compression stream. 


Decompressing LOBs 

Compressing data makes LOBs unread- 
able to other applications, so you need a 
function to decompress the data. Listing 2 
contains a BLOB decompression function 
that’s written in C#. This function follows 
the same principles used in the compression 
function, but instead of writing data to a 
stream and returning a compressed block, 
it reads from a stream and returns a decom- 
pressed block. 

As callout A in Listing 2 shows, the code 
uses a loop to read data from the Deflate- 
Stream object. This loop is necessary because 
the deflator doesn’t know how much data it 
needs to read or how much compression was 
achieved. The code loops until all bytes are 
read, then writes the bytes read in each pass 
to a MemoryStream object. 


Using the Functions 
The T-SQL code in Listing 3 loads the 
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BLOBs and CLOBs 


> UISTING 2 The Decompression Function 


using System; 

using System.Data; 

using System.Data.SqlClient; 
using System.Data.SqlTypes; 

using Microsoft.SqlServer.Server; 
using System.10; 

using System.10.Compression; 


public partial class UserDefinedFunctions 
{ 


[CMicrosoft.SqlServer.Server.SqlFunction(IsDeterministic 
DataAccess DataAccessKind.None) 1] 


public static SqlBytes fn_decompress(SqlBytes compressedBlob) 
{ 


true, 


if (compressedBlob.IsNull) 
return compressedBlob; 

// Prepare to read the data from the compressed stream. 

DeflateStream decompressor new DeflateStream(compressedBlob. Stream, 
CompressionMode.Decompress, true); 


@ // Initialize the variables. 
int bytesRead = 1; 
int chunkSize = 10000; 
byteL] chunk = new byteLCchunkSizeJ; 
// Prepare the destination stream to hold the decompressed data. 
MemoryStream decompressedData = new MemoryStream(); 
try 
{ 
// Read from the compressed stream. 
while ((bytesRead = decompressor.Read(chunk, @, chunkSize)) > Ø) 
{ 
// Write the decompressed data. 
decompressedData.Write(chunk, 0, bytesRead); 
} 
catch (Exception) 
throw; 
} 
finally 
{ 
// Clean up. 
decompressor.Close(); 
decompressor = null; 
D 
// Return a decompressed BLOB. 
return new SqlBytes(decompressedData); 
} 
wE 


PP USTING 3 Code That Loads the Assembly and Creates the Functions 


CREATE ASSEMBLY CBlobCompression] 
FROM 'D:\Development\BlobCompression.DLL' 
WITH PERMISSION_SET SAFE 
CREATE FUNCTION Cfn_decompress] (a@compressedBlob varbinary(MAX) ) 
RETURNS varbinary(MAX) AS EXTERNAL NAME 
CBlobCompression].CUserDefinedFunctions1.Cfn_decompress1]; 
CREATE FUNCTION Cfn_compress] (adblob varbinary(MAX)) 
RETURNS varbinary(MAX) AS EXTERNAL NAME 
CBlobCompression].CUserDefinedFunctions1.Cfn_compress]; 


PP LISTING 4 Sample Script 


CREATE TABLE #temp (blob_col varbinary(max)); 

INSERT INTO #temp 

VALUES(CONVERT(varbinary(max), 'To run your project, please edit 
the Test.sql file in your project. This file is located in the Test Scripts 
folder in the Solution Explorer.')); 

UPDATE #temp 

SET blob_col master.dbo.fn_compress(blob_col); 

SELECT CONVERT(varchar(1000), master.dbo.fn_decompress(blob_col)) 
FROM #temp; 

DROP TABLE #temp; 
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assembly in SQL Server 2005 (a process 
called cataloging during which the assembly 
is verified for security and reliability) and 
creates the compression and decompression 
functions. With the assembly registered and 
the functions created, you can compress and 
decompress data without having to modify 
your application layers or database objects. 

The sample script in Listing 4 demon- 
strates how you might use the compression 
and decompression functions in T-SQL 
code. This script first creates a table and adds 
some values to it. The script then runs a 
compression update followed by a SELECT 
statement that returns the uncompressed 
data. Note that this script allows column- 
level compression but lacks consistency 
check functions. 


You Don’t Need to Be Afraid 
Any Longer 
With the new varbinary(max),nvarchar(max), 
and varchar(max) data types, you can use 
SQL Server 2005's CLR to easily compress 
and decompress LOBs. All you need to do 
is write the compression and decompression 
functions, load the assembly, and create the 
functions. You're then ready to use those 
functions in your scripts. However, for 
large implementations or mission-critical 
applications, you might consider using third- 
party products, such as SQLCompress.NET 
(http://www.sqlcompress.net). In these situ- 
ations, third-party applications offer several 
advantages, including the use of algorithms 
rather than the Buffer property to get the 
data in and out of BLOBs (thereby avoiding 
the size limitation in pre-SP2 servers), veri- 
fication of compressed data to help detect 
data corruption, configurable memory 
block sizes (which keeps memory utiliza- 
tion down without impacting compression 
performance), and extensibility. SoL] 
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Advanced 


DATETIME 
Calculations, Part 3 


Calculate last/next occurrences 
of 3 weekday 


| my February and March columns, I talked about challenges related to datetime 
nN calculations. I presented techniques for separating the date and time parts of a 
datetime value, returning the first/last day of the month, and calculating a language- 
independent weekday number. Now, I want to offer some techniques for calculating 
the date of the last/next occurrences of a weekday related to a specific date (call 
it the event date)—for example, calculating the date of the most recent Monday 
related to today. 

All the techniques I present in this article will be based on the calculations of a 
language-independent weekday number, which I discussed last month. As a reminder, 
I presented two techniques to calculate a language-independent weekday number. 
One technique was based on calculating the offset in terms of days between a base 
date and the event date (call it diff). The base date had the same weekday as the one 
you wanted to set as the logical first day of the week (e.g., the string 19000101 or 
the integer 0 for Monday). When converting the integer 0 to a datetime value, you 
get the base date January 1, 1900, which happens to be a Monday. So 0 represents a 
date that falls on a Monday, 1 represents a date that falls on a Tuesday, and so on. The 
expression diff % 7 + 1 produced the weekday number. For example, to calculate 
the weekday of today’s date, assuming Monday as the first day of the week, you use 
the expression 


SELECT DATEDIFF(day, Ø, GETDATE()) % 7 + 1; 


If today happens to be a Tuesday, the above expression would return 2. In this article, 
I refer to this technique as datediff-based. 

Another technique I presented was based on neutralizing the impact of the 
DATEFIRST setting on the DATEPART calculation. The expression I used added 
@@DATEFIRST days to the event date and subtracted a constant representing 
the logical first day of the week that you want to use. For example, to calculate the 
weekday of today’s date, assuming Monday as the first day of the week, you use the 
expression 


SELECT DATEPART (weekday, GETDATE() + @aDATEFIRST - 1); 
Again, if today happens to be a Tuesday, the above expression would return 2. In this 


article, I refer to this technique as datepart-based. 


Last Occurrence of a Weekday 

As long as the datediff-based and datepart-based expressions are completely clear to you, 
we can proceed. You're now equipped to write expressions that calculate the last or 
next occurrence of a weekday. 
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Datediff. Suppose today’s date is 
December 19, 2006 (a Tuesday), and you 
want to calculate the last occurrence of 
Monday, which might be today (that is, 
the calculation is inclusive). So if today is 
a Monday, the expression would return 
today’s date. Because I’m assuming in my 
example that today’s date is December 
19, 2006 (Tuesday), the expression should 
return the most recent occurrence of 
Monday, which is yesterday. Here’s the 
datediff-based expression that returns the 
date of the most recent Monday: 


-- Last Monday (inclusive) 

SELECT DATEADD(day, DATEDIFF 
(day, 0, -- Base Monday date 
GETDATE()) /7*7, Ø); -- Base 
Monday date 


The DATEDIFF function calculates the 
difference in terms of days between a base 
date, which is a Monday, and today’s date 
(call it diff}. The expression then divides 
diff by 7 and multiplies by 7, practically 
subtracting the number of days that passed 
since the most recent Monday (call the 
result floored_diff). Finally, the expression 
adds floored_diff days to the base date, 
returning the date of last Monday. 

As expected, the expression returns 
last Monday’s date, assuming today is 
December 19, 2006: 


2006-12-18 00:00:00.000 


As a side note, to validate the expres- 
sions in this article, you can explicitly 
specify 20061219 as the date instead of 
GETDATE(). 

To return the most recent Tuesday’s 
date, provide a date that falls on a Tuesday 
as the base date (represented by the 
integer 1 or the string 19000102): 
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SELECT DATEADD(day, DATEDIFF(day, 
1, GETDATE())/7*7, 1); 


You'll get today’s date (December 19, 
2006) as output because the calculation is 
inclusive: 


2006-12-19 00:00:00.000 


To return the most recent Sunday’s date, 
provide a date that falls on a Sunday as the 
base date (represented by the integer 6 or 
the string 19000107): 


SELECT DATEADD(day, DATEDIFF(day, 
6, GETDATE()) /7*7, 6); 


-- Next Monday (Inclusive) 
SELECT DATEADD(day, DATEDIFF(day, 0, 
GETDATE()-1) /7*7 + 7, 0); 


Output: 
2006-12-25 00:00:00.000 


-- Next Tuesday (Inclusive) 
SELECT DATEADD(day, DATEDIFF(day, 1, 
GETDATE()-1) /7*7 + 7, 1); 


Output: 
2006-12-19 00:00:00.000 


-- Next Sunday (Inclusive) 
SELECT DATEADD(day, DATEDIFF(day, 6, 
GETDATECO)=13 ZTS + 7; 6); 


Output: 
2006-12-24 00:00:00.000 


-- Next Monday (Exclusive) 

SELECT DATEADD(day, DATEDIFF(day, 0, 
GETDATE()) /7*7 + 7, D; 

Output: 


2006-12-25 00:00:00.000 


-- Next Tuesday (Exclusive) 

SELECT DATEADD(day, DATEDIFF(day, 1, 
GETDATEQ)) /7*7 + 7, 1) 

Output: 

2006-12-26 00:00:00.000 


-- Next Sunday (Exclusive) 

SELECT DATEADD(day, DATEDIFF(day, 6, 
GETDATE()) /7*7 + 7, 6); 

Output: 

2006-12-24 00:00:00.000 


> > FIGURE | Calculation of next 


weekday occurrence 


22 April 2007 


You get the output 
2006-12-17 00:00:00.000 


Another flooring technique that you can 
use instead of dividing diff by 7 and then 
multiplying it by 7 is to subtract from diff a 
number representing the offset (in terms of 
days) of “today’s day of the week” from “the 
day of the week you need.’ For example, 
if the requested day of the week is Tuesday 
(represented by the integer 1 or the string 
19000102), this offset would be expressed as 


DATEDIFF(day, 1 /* Base Tuesday 
Date */, GETDATE()) % 7 


Call this expression offset. Now, embed offset 
as part of an expression that calculates the 
most recent occurrence of a requested day 
of the week: 


-- Don't run 

SELECT DATEADD(day, DATEDIFF(day, 
Ø /* Base Date */, GETDATE()) - 
- diff - offset, Ø /* Base Date 
*/); 


In this case—unlike the expression calcu- 
lating offse-—the base date you use doesn’t 
really matter, as long as you specify the same 
base date in both the DATEDIFF function 
and the DATEADD function. The complete 
expression looks like 


-- Last Tuesday (inclusive) 

SELECT DATEADD(day, DATEDIFF 
(day, Ø /* Any Base Date 
*/, GETDATE()) -- diff - 
(DATEDIFF(day, 1 /* Base 
Tuesday Date */, GETDATE()) % 
7), -- offset Ø /* Any Base 
Date */); 


If you're after the most recent Sunday’s date, 
all you need to do is specify a Sunday base 
date (the integer 6 or the string 19000107) 
instead of a Tuesday base date: 


-- Last Sunday (Inclusive) 

SELECT DATEADD(day, DATEDIFF(day, 
Ø /* Any Base Date */, 
GETDATE()) - (DATEDIFF(day, 

6 /* Base Sunday Date */, 
GETDATE()) % 7), Ø /* Any Base 
Date */); 
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Datepart. You can also rely on the 
datepart-based technique to calculate offset. 
Remember that the datepart-based tech- 
nique calculates a language-independent 
weekday number. If you think about it, offset 
is nothing more than a weekday number 
minus one, assuming the weekday you're 
looking for is the first day of the week. The 
following expression uses the datepart-based 
technique to calculate a language-indepen- 
dent weekday number, assuming Monday is 
the first day of the week: 


DATEPART (weekday, GETDATE() + aa 
DATEFIRST - 1 /* datefirst is 
Monday */) 


Don't confuse this expression’s constant (1 
for Monday) with the constants in the date- 
diff-based technique. This time, the constant 
doesn’t represent a base date as an integer; 
rather, it represents the logical first day of the 
week you want to set—1 for Monday, 2 for 
Tuesday, and so on. So, to get offset (weekday 
number minus one), assuming Monday is the 
first day of the week, use the expression 


DATEPART (weekday, GETDATE() + 3a 
DATEFIRST - 1 /* datefirst is 
Monday */) - 1 


To get offset assuming Tuesday as the first day 
of the week, subtract the constant 2 from 
@@DATEFIRST: 


DATEPART (weekday, GETDATE() + aa 
DATEFIRST - 2 /* datefirst is 
Tuesday */) - 1 


Finally, embed the new offset calculation in 
the complete expression that returns the last 
occurrence of a weekday—for example, the 
most recent Tuesday: 


-- Last Tuesday 

SELECT DATEADD(day, DATEDIFF(day, 
Ø /* Base Date */, GETDATE()) - 
(DATEPART(weekday, GETDATE() + 
@aDATEFIRST - 2 /* datefirst is 
Tuesday */) - 1), Ø /* Base 
Date */); 


To return last Sunday’s date, subtract the 
constant 7 from @@DATEFIRST: 


-- Last Sunday 
SELECT DATEADD(day, DATEDIFF(day, 
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Ø /* Base Date */, GETDATE()) - 
(DATEPART(weekday, GETDATE() + 
@aDATEFIRST - 7 /* datefirst is 
Sunday */) - 1), Ø /* Base 
Date */); 


I find this article’s first datediff-based 
technique to be the most elegant way to 
calculate the last occurrence of a weekday. 
Doubtless, it’s the shortest. So, from this 
point on, I'll rely on this technique for other 
calculations as well. As a reminder, here’s the 
expression I used to calculate the last occur- 
rence of a Monday: 


SELECT DATEADD(day, DATEDIFF(day, 
Ø, GETDATE()) /7*7, 0); 


Twice, you specify a base date representing 
the weekday you're after (the integer 0 or the 
string 19000101 for a Monday base date). 

Suppose you need the calculation of 
the last occurrence of a weekday to be 
exclusive—not to take the event date into 
consideration. All you need to do is sub- 
tract 1 from the event date. For example, 
to produce the most recent occurrence of 
Monday in an exclusive manner, you would 
use the expression 


-- Last Monday (exclusive) 
SELECT DATEADD(day, DATEDIFF(day, 
Ø, GETDATE()-1) /7*7, Ø); 


This method is applicable to all the tech- 
niques that I demonstrated earlier. Simply 
subtract 1 from the event date to make the 
calculation exclusive. 


Next Occurrence of a Weekday 

If you need to produce the next occurrence 
of a weekday corresponding to a given event 
date, you can rely on techniques that are 
similar to those I presented for last occur- 
rence. You might think that to produce 
the next occurrence of a weekday, all you 
need to do is add 7 days to the calculation 
of the last occurrence of that weekday. But 
the calculation is trickier than you think. 
To make the calculation of the next occur- 
rence inclusive, you need to add 7 days to 
the exclusive calculation of the last occur- 
rence. Think about it: If today is Tuesday, the 
inclusive calculation of the last occurrence 
of Tuesday will yield today’s date. When you 


Solution to March’s Puzzle: Free Tuna 
You go to the grocery store and grab eight cans of tuna from the shelf. You go 
to the cash register to pay. In a good mood, the store owner hands you three 
plastic bags and says, “If you can arrange the eight cans in these three plastic 
bags such that each bag contains an odd number of cans, you can have them 
for free.” Can you think of a way to get that free tuna? 

Obviously, you can’t divide the eight tuna cans into three separate plastic 
bags so that each holds an odd number of cans. However, nothing in the puzzle 
dictates the arrangement of the bags around the tuna cans. The sum of three odd 
numbers xt+y+z, where each number is considered only once, naturally amounts 
to an odd number. However, taking one of the odd numbers into consideration 
twice allows for an arrangement in which one of the elements is even (say, 
y)—for example, (y+(x))+(z) = 8. The use of parentheses is intentional—each 
pair of parentheses represents a plastic bag. For example, let x equal 1, y equal 
2, and z equal 5: You place 1 tuna can in plastic bag A, 2 tuna cans in plastic 
bag B, and 5 tuna cans in plastic bag C. Then, place plastic bag A in plastic bag 
B. You end up with 1 tuna can in bag A, 3 in B (x+y), and 5 in C. 

As an aside, if you like trying to solve open puzzles, the tuna cans puzzle 
reminds me of a mathematical conjecture that so far hasn’t been proven. The 
conjecture is named after its conjurer—“Goldbach’s conjecture.” The original 
conjecture says, Every odd number can be expressed as the sum of three prime numbers. 
Euler simplified the conjecture to the form Every even number can be expressed 
as the sum of two prime numbers. In case you're wondering, I’m not planning on 
publishing the proof to this conjecture next month. 


April’s Puzzle: Naming an Heir 
A mighty king had three sons and wanted to declare the wisest of them as his 
heir. He decided to give them a logic puzzle to test their wisdom. He placed 
the sons in a triangular room, each in a different corner, and placed a hat on 
each son’s head. The king said, “You need to determine the color of your hat. 
You can’t take your hat off to look at it, and you can’t communicate in any way. 
The hat on your head is either green or red. At least one of you is wearing a 
green hat. I'll be waiting outside the door and will ring a bell every 5 minutes. 
You can’t leave the room until you know the color of your hat. If you know 
the answer, you must wait for the next bell ring, then come tell me the answer.” 
At the third bell ring, one of the sons opened the door and told the king the 
answer. The king said, “You're correct, and I’m naming you my heir. However, 
Im disappointed in you. You still have much to learn.” What was that son’s 
answer, and why was the king disappointed? 
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add 7 days, you end up with a Tuesday date 
a week ahead of today—not with today’s 
date. However, if you use the exclusive 
calculation of last Tuesday’s date, you will 
get the Tuesday date a week ago. Adding 7 
days gets you today’s date, effectively making 
the calculation of the next occurrence of a 
weekday inclusive. Similarly, to make the 
calculation of the next occurrence exclu- 
sive, you need to add 7 days to the inclusive 
calculation of the last occurrence. 
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To clarify this idea, Figure 1 shows a 
few examples, assuming today’s date is 
December 19, 2006 (Tuesday). As you can 
probably figure out, instead of adding 7 days, 
you can add (or subtract) any multiplication 
of 7 days to get the next/last occurrence of 
a weekday several weeks ahead/ago. 


Messing with Your Head 
You might think my goal is to mess with 
your head. Unfortunately, there are no built- 
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in functions that perform such calculations, 
and the techniques I’m sharing are as simple 
as you can get. Because calculations such as 
the last/next occurrence of a weekday are 
frequently needed, it’s good to be familiar 


with the techniques I’ve presented. But 
were not done yet. Next month, Pll dis- 


cuss datetime-related calculations further. 


If you think the calculations in this article 
were tricky, just wait. In the meantime, get 


involved! Do you have datetime solutions 
you'd like to share? Check out the sidebar 
“Share Your DATETIME Thoughts” for 
some great reader responses. SOL 
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Share Your DATETIME Thoughts 


We received some interesting feedback to Itzik’s series about datetime calculations. Remember that you can share 
your thoughts about any of our articles in the SQL Server forums (http://sqlforums.windowsitpro.com), and you 
can leave comments specifically about Itzik’s columns in the T-SQL subforum (http://sqlforums.windowsitpro.com/web/ 
forum/categories.aspx?catid=80&entercat=y). We thought we'd share a couple of responses that Itzik has received from 


readers. 
Reader Mike Smith shared the following quick technique: 


Inspired by your article “DATETIME Calculations, Part 1” (InstantDoc ID 94487), I thought binary manipulation of a datetime 
value would offer the best performance when trimming the date or time part. I was unable to find a way (e.g, bitwise operators, reverse 
function) to grab the date portion, but I did discover this statement that keeps the time portion, and which I believe would offer fantastic 
performance (and it’s simple!): 


select cast(cast(adt as varbinary(4)) as datetime); 


Itzik responds, “That’s a great technique for extracting only the time! You could use similar logic to extract only the 
date: 


select cast(substring(cast(getdate() as binary(8)), 1, 4) + Ox00000000 as datetime) 


but that requires further manipulation, making it a bit cumbersome.” 
Reader J. Ashley Bryan also shared a simple, intuitive technique: 


In your article, you review three techniques for trimming the date value out of a datetime datatype. There is a fourth technique that I 
believe warrants consideration. You start by casting the datetime value to a float. The date and time are now represented as a decimal value, 
with the day to the left of the decimal and the time to the right. You then apply the Floor function against the float value to get the first 
whole number less than the datetime decimal value. With the time portion of the datetime value zeroed out, you then cast the float value 
back to datetime. The result is the midnight datetime entry for the value submitted. 


select cast(floor(cast(getdate() as float)) as datetime); 


Using the same performance test you used, I added this technique to compare the results. In a_five-run comparison, the float technique had 
performance comparable to that of your recommended third technique and posted slightly better results than that technique overall. I used 
the results of your worst-performing technique (the first one) as the baseline for the comparative percentages, as you see in Table A. 

The float technique is a bit more intuitive and readable than the others, while maintaining good performance results. In addition, this 
technique can be used in other areas to help perform date-range operations by using the Ceiling function to find the next highest whole 
number from the value submitted. 


..-where dbdate >= cast(floor(cast(getdate() as float)) as datetime) and dbdate < 
cast(ceiling(cast (getdate() as float)) as datetime); 


Itzik responds, “I’m usually reluctant to use floats because of their imprecise nature, particularly in calculations that 
require complete accuracy. However, I tested the technique with both midnight values and last possible accuracy unit (i.e., 
23:59:59.999), and it seemed to perform well. Thanks for sharing!” sou 
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by Tyler Chessman 


New features boost Excel’s 
ability to analyze SQL Server 
Analysis Services OLAP cubes 


| the Microsoft business intelligence (BI) platform, the SQL Server 
nN Analysis Services (SSAS) 2005 OLAP engine is the component 
of choice for summarizing and presenting vast amounts of data. On 
the desktop, though, Microsoft Excel is typically the tool of choice for 
analyzing, organizing, and making sense of data. It’s no surprise, then, 
that most OLAP tools offer some type of integration with Excel. The 
built in PivotTable (and PivotChart) reports in Excel have tradition- 
ally provided one way of browsing OLAP cubes—albeit with certain 
restrictions and missing functionality. Pll highlight several key new 
PivotTable enhancements that make Microsoft Office Excel 2007 a 
much better client for SSAS 2005 OLAP cubes. (I'll discuss Excel 
2007's integration with SSAS’s data-mining features in an upcoming 


article.) 
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ton Ges on the Network 
AdventureWorks Data Conmection Library 
DCL Folder 


Comnecson Ses 


on this compute: 


MSH MoneyCentral Investor Currency Rates 
[Bank] 

MSN Haney( entrad investor Major Inckcoes 
[Mark] 

MSH MoneyCentral Investor Stock Quotes 
[Bark] 


> > FIGURE | Excel 2007’s connection- 
management feature 


PivolTable Evolution 

PivotTable reports make it possible to 

aggregate large amounts of data across 

different categories of interest and to 
more easily analyze, explore, and present 

this data. When Microsoft Excel 2000 

was introduced, users had the option of 

using an OLAP cube as the data source 
for a PivotTable. This feature let users 
analyze much greater volumes of data 
than they could in a spreadsheet and also 
allowed much of the CPU- and memory- 
intensive computation to occur on the 
server. However, as an OLAP client, Pivot 

Tables had two major drawbacks: 

e Many of the advanced SSAS fea- 
tures—such as drill-through/actions, 
member properties, and server-side 
formatting—weren’t recognized or 
available. 

e PivotTables are effectively “locked.” In 
other words, you can’t insert additional 
rows or columns in a PivotTable, and 
referencing PivotTable data in other 
spreadsheet cells is problematic. 


Microsoft addressed some of these drawbacks 
when it introduced the Excel Add-in for SQL 
Server Analysis Services, a free download for 
Microsoft Office Excel 2003 and Excel 2002 
that offers a Pivot TableJike experience—but 
with better SSAS support and integration 
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with “native” Excel functionality. 
(For more information about 
the add-in, see “Integrating SQL 
Server & Office 2003,” May 2006, 
InstantDoc ID 49688 and the 
Web-exclusive article “Microsoft 
Accelerates in BI Space,’ June 
2004, InstantDoc ID_43076.) In 
Excel 2007, most of the features 
in the Excel add-in are incor- 
porated directly into the built-in 
PivotTables, along with many 
new features. Let’s take a closer 
look at some of those features. 


Getting Started: 
Establishing a 
Connection 

To work through the examples 
in this article, make sure you 
have access to an instance of 
SSAS 2005 with the Adventure Works 
sample SSAS database. This sample database 
isn’t installed by default. You'll 
need to download the database 


3x) 


the PivotTable command at the far left of 
the Rubbon; click this command to display 
the Create PivotTable Window. Select the 
Use an external data source option, then click 
Choose Connection. 

Here you'll see the first major enhance- 
ment to Excel in the form of enhanced con- 
nection-management capabilities, as Figure 
1 shows. In addition to seeing connections 
specific to the worksheet and your computer, 
you'll also see a section called Connection files 
on the Network. This set of connections is cre- 
ated from an Excel Services Data Connection 
Library (DCL) on a Microsoft Office Share- 
Point Server 2007 site (Excel Services delivers 
Excel as a thin client through SharePoint). 
Setting up (and distributing) connections on 
an end-user machine can take a bit of work, 
so—if your organization deploys SharePoint 
2007—these network connections will help 
reduce complexity. 

Go ahead and click Browse for More, 
then click New Source. Select Microsoft 


(included with several other tee oie we Fietd 

sample databases) at http://www __Aa 7 h 
-microsoft.com/downloads/ |, — 6 D Pivot Table Field List 
details.aspxefamilyid=e719ecf7- |a[——~T Sow fede rested 5: 
9f46-4312-af89-6ad87 = | mei | fe wae 
O2e4e6e&displaylang= |5 Jobus reoot choose j penosa 

en and follow the instal- |% fields tromthe pwotrable [End of Dar Ra 
lation instructions at http:// : Gast 3 E Prencel Reporting 
msdn2.microsoft.com/en-us/ | Cam 
library/ms143804.aspx or from |“ Sees = ages 

the SQL Server 2005 CD- | & = E ee 
ROM. I also recommend you |` 2 = sen 

apply SQL Server 2005 Ser- | ss = ag Mere fies 

vice Pack 2 (SP2) to the SSAS | * 3 [J ome 

instance so that all Excel 2007's a ng 
features will be enabled. (See the | * a: (yore fekis 

SQL Server 2005 SP2 Readme |2 Ee pven 

file for more information about |» Sia OERE E T 
SP2’s support of Microsoft Office = T magenta G Coens 
2007 system BI features.) Then, | % 

open Excel 2007 and click the |% E Rom Labele = we 


Insert Ribbon tab. Note that | 2 
most Office 2007 applications 
(Microsoft Office Outlook 2007 |: 


149% Deamplel 3 A 


Defer Layout Update 


is a notable exception) now use 


Ready 


a UI element called the Ribbon, 
which replaces the traditional 
menus and toolbars. You'll see 
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> > FIGURE 2 Creating a new PivotTable to work 


with data from the AdventureWorks database 
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then click New Source. Select Microsoft 
SQL Server Analysis Services as the data 
source. Enter your server name.Then, in the 
Select Database and Table window, select the 
Adventure Works DW database, then select 
the Finance perspective. In the Save Data 
Connection File and Finish window, click 
Finish to conclude the wizard, then click 
OK in the Create PivotTable window. You 
should now see a PivotTable in your work- 
sheet and a PivotTable Field List, as Figure 
2 shows. At the top of the Excel Ribbon, 
you'll also see the text Pivot Table Tools (if you 
don’t see this text, make sure the PivotTable 
has been selected in your worksheet). When 
you click the text, you'll see the Ribbon’s 
Options tab, which contains different Pivot- 
Table and PivotChart commands. 


An Enhanced Field List 

The first thing you might notice in the 
PivotTable Field List is the built-in support 
for measure groups—a new feature in SSAS 
that allows cubes to be built from multiple 
fact tables. In our example, the Exchange 
Rates and Financial Reporting measure 
groups are separately displayed at the top 
of the field list; you can also filter the entire 
field list (by using the Show fields related to 
drop-down box at the top of the list) to 
display only the measures, dimensions, or 
Key Performance Indicator (KPIs) related 
to a specific measure group. The field list 
also recognizes dimension display folders 
(e.g., notice the Calendar and Fiscal folders 
in the Date dimension), named sets (e.g., 
the Summary P&L Set in the Account 
dimension), and KPIs. At the bottom of 
the field list, you'll see four different areas: 
Report Filter, Column Labels, Row Labels, 
and Values. These areas represent a new way 
of authoring reports. 


A New Report-Authoring 
Experience 

In the field list’s Financial Reporting 
measure group area, look at the Amount 
measure. Notice that Amount shows up in 
the PivotTable. If you’ve applied SP2 to the 
SSAS instance, notice also that the server- 
side formatting (in this case, currency) 1s 
recognized. Amount also appears at the 
bottom of the field list in the Values area. 
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Next, in the Account dimension, check 
the Accounts hierarchy; the hierarchy will 
appear in the PivotTable (as rows) and in 
the field list’s Row Labels area. Finally, 
expand the Date dimension’s Fiscal folder 
and check the Date.Fiscal hierarchy; it will 
appear in the PivotTable (as columns) and 
in field lists Column Labels area. If you 
followed me so far, your PivotTable and 
field list should look like those in Figure 
3. (Note that I’ve rearranged my field list 
by using the Fields Section and Areas Section 
Stacked view so that the areas are displayed 
side by side with the field section.) 

Next, try to add and rearrange fields 
on the report. Unlike the old PivotTable 
drag-and-drop experience in Excel 2003, 
you now add or remove fields by simply 
checking or unchecking them. Likewise, to 
rearrange a field in a report (e.g., move a 
field from a row to a column), you simply 
drag and drop a field between the areas in 
the field list. Although Im experienced 
with PivotTables in Excel 2003, I’ve very 
quickly come to prefer this new report- 


authoring approach. But if you decide you 
like the “old way” of doing things, you can 
right-click a PivotTable, select the Pivot- 
Table Options menu item, then check the 
Classic PivotTable layout option in the 
Display tab. While you're in this Options 
window, you might also want to select the 
Show calculated members from OLAP server 
check box to ensure that server-defined 
calculated members are visible. 

Another change in PivotTable reports 
has to do with the default row layout 
when navigating a dimensional hierarchy 
(or when displaying two or more dimen- 
sions on a row). For example, if you expand 
the Balance Sheet dimension member (to 
expand a dimension, click the + sign next 
to the dimension name in the PivotTable 
report), notice how the two child dimen- 
sions (Assets and Liabilities and Owners 
Equity) are indented—but still remain in 
the same column. This feature lets you 
navigate hierarchies without additional 
columns being allocated—nice for main- 
taining screen real estate and/or when 


A Ts 5 E F 
1 
2 
3 Amount Fiscal Year x 
4 AccountLevel01 |v] +#FY 2002 + FY 2003 + FY 2004 + FY 2005 Grand Total 

# Balance Sheet $0.00 l $0.00 $0.00 $0.00 
6 | #Net Income $3,250,075.00 $4,606,605.00 $4,752,823.00 $12,609,503.00 
7 |#Statistical Accounts NA NA NA NA NA 
8 PivotTable Field List _ zi l i > x| 
9 “4. 
10 Show fields related to: Gay 
11 (all) ~| W Report Filter 
12 [E = Financial Reporting = || 
13 Amount ` 
14 -N = 
15 3 Ẹ es = HE Row Labels | 
16 = E] Account | Accounts -| 
17 = [M Accounts | 
18 E Gy Sets | 
19 C Summary P&L a Column Labels | 
20 Œ Gay More fields | Date. Fiscal -l 
= |E Ë Date | 

= Q Calendar | 

5 E E Fiscal = Values 
zi = [M Date.Fiscal | Amount -| 
= (Date. Fiscal Quarter of Year | | 
= (Date. Fiscal Semester of Year | 
28 (Date. Fiscal Year sf [C] Defer Layout Update Update 


FIGURE 3 PivotTable containing AdventureWorks data and PivotTable field list 
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you've defined other Excel calculations to 
the right of your PivotTable. 


OLAP Formulas 


At the beginning of the article, I mentioned 
that one drawback of PivotTables is they’re 
“locked.” This is still the case with Excel 
2007; you can’t insert rows or columns into 
a PivotTable. But you can now convert a 
PivotTable to a set of OLAP formulas— 
which will then let you take full advantage 
of Excel’s free-form capabilities. (To convert 
the PivotTable, from the Ribbon’s Options 

Tab, select the OLAP tools command, then 

select Convert to Formulas.) Figure 4 shows a 

PivotTable that I’ve converted. Notice that 

the selected cell, B8, is defined using the 

CUBEVALUE function. There are seven 

new CUBE functions in Excel 2007 that 

let you fetch dimension members, measure 
values and member properties, and perform 
other related tasks. Notice also the Gross 

Margin% row (row 9), doesn’t come from 

the cube; rather, I inserted this row and used 

standard Excel calculations to reference the 

OLAP-based Gross Sales and Gross Margin 

values in each respective column. 

Here are three other important points 
about OLAP formulas: 

e Notice in Figure 4 that the row filter 
(Date. Fiscal Year) still has a filter symbol 
in its value cel. When you convert a 
PivotTable to OLAP formulas, you have 
the option of retaining the row filters. 
This option gives you the best of both 
worlds in that you can customize your 
worksheet (e.g., insert new columns 
and rows) while still having the ability 
to filter the OLAP data by different 
dimension members. 

e You don’t have to start with a PivotTable 
to use OLAP formulas. Rather, you 
can use these formulas on a cell-by-cell 
basis. 

e Converting a PivotTable to a set of 
OLAP formulas is a one-way operation; 
you can’t revert back to a PivotTable. 
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58 X fe COUBEVALUE("AdventureWorks_FinancePerspective",S851,5453,5A45,6854)} 

Pi A c D E F 

1 Oate.Fiscal Year FY 2004 a 

= 

3 Amount Fiscal Quarter of Year 

4 Summary P&L Fyal FYQ2 FY Q3 FY Q4 Grand Total 

5 Net income $1,968,644.00 $1,270,239.00 $391,875.00 $1,122,065.00 $4,752,823.00 
6 Operating Profit $2,418,156,00  $1,627,150.00 $658,987.00  $1,457,895,90 $6,162,228.90 
7 Operating Expenses $2,849, 859.00  $2,825,440.00 $2,806,471.00  $2,849,543.10 $11,331,313.10 
| |Gross Margin $4,452,630.00 53,465,458.00 $4,307,435,00 $17,493,542.00 
9 GrossMargin% 63.6% 67.6%" 65.9%" 66.2%" 65.79) 
10 Net Sales $7,546,239.00 $6,408,354.00 $5,122,284.00 $6,256,874.00 $25,333, 751.00 
11 Gross Sales $8,277,981.00 $6,585,690.00 $5,262,492.00  $6,506,493.00  $26,634,656.00 

2 Returns and Adjustmer $270,175.00 $158,837.00 $133,337.00 $197,364.00 $759,713.00 
13 Discounts $461,567.00 $18,499.00 $5,371.00 $54,255.00 $541,192.00 
14 Total Cost of Sales $2,278,224.00  $1,955,724.00 $1,656,826.00 $1,949,435.00 $7,840,209.00 
15 Other income and Expe $3,095.00 $3,062.00 $3,074.00 $3,926.10 $15,157.10 
16 Interest Income $10,934.00 $10,812.00 $10,836.00 $11,390.00 $43,974.00) 
17 interest Expense $16,811.00 $16,620.00 $16,655.00 $17,510.00 $67,596.00 
18 Gain/Loss on Sales of A ($13,450.00) ($13,297.00} {513,326.00} ($14,011.00) {554,084.00 
19 Other Income $22,422.00 $22,167.00 $22,217.00 ($27,953.90) $38,852.10 
20 Curr Xchg Gatn/iLoss} $57,011.00 $57,011.00 
22 Taxes $452,607.00 $360,013.00 $270,186.00 $344,757.00 $1,427,563.00 


> > FIGURE 4 PivotTable converted to OLAP formulas 


More Great Features... and a 
Missing One 

Several other PivotTable features that are 
new in Excel 2007 improve upon Excel's 
OLAP integration. They include 

e Member-property support. Member prop- 
erties (which are now simply referred to 
as attributes in SSAS 2005) are visible 
when hovering over a dimension mem- 
ber. Attributes let you see additional 
information about a dimension member 
(e.g., a phone number for a customer). 
You can optionally choose to have attri- 
butes displayed in a PivotTable report 
alongside the dimension member. 

KPI support. Excel 2007 can display 
the value, goal, status, and trend for a 
KPI—including the graphical indicator 
defined for a KPI’s status and trend. 
Support for actions. Drill-through is now 
considered a cube action, and Excel 
2007 does a nice job of returning drill- 
through results into a new, formatted 
worksheet. 

Enhanced filtering. With SSAS 2005 
SP2 applied, you can filter dimension 
members according to date, label, and 
measure values. 


Notwithstanding all the new PivotTable 
features, one that’s missing is the ability to 
create calculated fields (what you’d think 
of calculated members with respect to an 
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OLAP cube). On the Options tab in the 
Excel Ribbon, there’s a command called 
Formulas (located in the Options tab’s Tools 
section); for OLAP reports, the Formulas 
command is disabled. Although OLAP 
Formulas provide a workaround, I’m still 
hoping calculated fields for OLAP will be 
enabled in a future service pack or release. 


OLAP and More 


Excel, in my opinion, will continue to be 
the desktop tool of choice for analyzing, 
organizing, and making sense of data. The 
OLAP-specific enhancements in Excel 
2007 greatly improve its usefulness as a 
means to analyze and manipulate data from 
OLAP cubes. In addition to the features 
I’ve covered here, Excel 2007 also offers an 
enhanced PivotChart experience—backed 
by a new graphics engine to provide better 
charting and visualization capabilities. For 
more examples, details, and discussions 
concerning Excel 2007 and SSAS, check 
out the Microsoft Excel 2007 blog at 
http://blogs.msdn.com/excel/archive/tags/ 

Sou 
InstantDoc ID 94270 


Tyler Chessman ttylerc@microsott.com) is a Micro- 
soft technology specialist who helps Microsofts database and 
business intelligence platform customers. Before joining Microsoft, 
he spent several years as a consultant implementing financial, 
analytical, and sales-force—automation solutions. 
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Use Missing-Index 
Grouns for Query 
Tuning 


Identify missing indexes that you 
could implement to improve query 
performance 


S Eog 2005 provides metadata about missing indexes through 

four dynamic management objects. Last month in “Uncovering 
Missing Indexes,’ InstantDoc ID 94884, we examined two of those objects, the 
sys.dm_db_missing_index_details view and the sys.dm_db_missing_index_col- 
umns function, and walked through sample queries that generated missing- 
index information to populate the dynamic management objects. Lets continue 
exploring the missing-index feature and discuss the other two missing-index 
dynamic management views, sys.dm_db_missing_index_group_stats and sys 
.dm_db_missing_index_groups, which provide information about missing-index 
groups. As with the other two dynamic management objects covered last month, you 
can use the information obtained from sys.dm_db_missing_index_group_stats and 
sys.dm_db_missing_index_groups to identify missing indexes that could be created to 
improve query performance. As you'll see, you can also use SQL Server 2005’s XML 
Showplan feature to view missing-index information in XML format. 


Mapping Indexes to Index Groups 

Missing indexes each belong to an index group, even if the group has only one index. 
In fact, in SQL Server 2005, you'll never find multiple indexes in a group. Each index 
group appears in the dm_db_missing_index_group_stats view along with information 
about how many times this missing-index group could have been used and what kind 
of performance improvement you might expect if the indexes in the group were cre- 
ated. Each missing-index group is identified by a unique value called a group_handle. 
Microsoft plans to include the ability to indicate indexes that need to be combined 
into a group in a future version of SQL Server.This capability is intended to deal with 
queries that require the use of multiple indexes for optimal performance. SQL Server 
2005 can provide recommendations based on only one index. 


Download the listing at 
InstantDoc ID 95220 
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To map the missing indexes that you 
saw last month in sys.dm_db_missing_ 
index_details to a missing-index group, 
so that you can obtain information about 
how useful such an index might be, 
the dm_db_missing_index_groups view 
serves as a mapping table to map index_ 
handle values to group_handle values. 
Because there are no index groups with 
multiple indexes in SQL Server 2005, 
you should see a one-to-one relationship 
between missing indexes and missing- 
index groups. To obtain the missing-index 
data, start by running the script in Listing 
1, page 30. This script creates the same 
three tables in the Adventure Works data- 
base that I used last month, then executes 
a SELECT query by using those three 
unindexed tables. 

Examining sys.dm_db_missing_index_ 
details should show you two missing 
indexes, just as we saw last month. We 
can now join sys.dm_db_missing_index_ 
details to the two other views by using the 
code below. 


SELECT * FROM 
sys.dm_db_missing_index_ 
details d 
JOIN sys.dm_db_missing_ 
index_groups g 
ON d.index_handle = 
g.index_handle 
JOIN sys.dm_db_missing_ 
index_group_ 
stats s 
ON g.index_group_handle 
=s.group_handle 


(Some code in this article wraps to mul- 
tiple lines because of space constraints.) 
The output is too wide to show here, 
but the key column values to look at for 
performance information are 


Kalen Delaney (kalen@insidesqlserver.com) has 
been working with SQL Server for 20 years and provides SQL 
Server training and consulting to clients around the world. 
Her most recent book is /nside Microsoft SQL Server 2005: The 
Storage Engine (Microsoft Press). 
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The missing-indexes feature gives you a head 


Start on query tuning. 


unique_compiles—shows the number of 
compilations and recompilations that 
would have benefited from this miss- 
ing-index group. These compilations 
don’t all have to be the same query; 
many different queries can potentially 
make use of the suggested indexes. 
user_seeks—shows the number of seek 
operations performed in user queries 
for which the recommended index in 
the group could have been used. 
user_scans—shows the number of scan 
operations performed in user queries 
for which the recommended index in 
the group could have been used. 
avg_user_impact—shows the average 
percentage improvement that user 
queries might have if the missing- 
index group were implemented. On 
average, all user queries (seeks and 
scans) that use an implemented miss- 
ing-index group would show in a 


LISTING | Creating 3 Tables and 
Running a Test Query 


USE AdventureWorks; 
GO 


IF EXISTS (SELECT 1 FROM sys.tables 
WHERE name = 'OrderHeader') 
DROP TABLE OrderHeader; 
GO 
SELECT * INTO dbo.OrderHeader 
FROM Sales.SalesOrderHeader; 
GO 
IF EXISTS (SELECT 1 FROM sys.tables 
WHERE name = 'Territory') 
DROP TABLE Territory; 
GO 
SELECT * INTO dbo.Territory 
FROM Sales.SalesTerritory; 
GO 
IF EXISTS (SELECT 1 FROM sys.tables 
WHERE name = 'Customers') 
DROP TABLE Customers; 
GO 
SELECT * INTO dbo.Customers 
FROM Sales.Customer; 
GO 
SELECT SalesOrderID, OrderDate, 
Status, h.CustomerID, 
c.AccountNumber, Name as 
Territory, CountryRegionCode as 
CountryRegion 
FROM OrderHeader h JOIN Customers c 
ON h.CustomerID = c.CustomerID 
JOIN Territory t 
ON c.TerritoryID = 
t.TerritoryID 
WHERE c.TerritoryID = 2; 
GO 
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decrease in this percentage in their 
execution cost. 


Of course, there are no guarantees about 
the actual improvement you'll realize if you 
build the suggested missing indexes. These 
values are all estimates derived by the SQL 
Server optimizer when it optimizes your 
queries. 


Missing-Index Data Is Missing 
The missing indexes feature is intended 
to give you a head start on query tuning 
by using SQL Servers own optimization 
process to generate a list of possible ways to 
improve your queries. However, not all que- 
ries, even those that might benefit greatly 
from a new index or two, will generate data 
in the missing-index dynamic management 
objects. 

At present, the missing-index informa- 
tion is generated only for queries that meet 
these conditions: 

e The query must have a predicate (Le., 
a WHERE clause) referencing a col- 
umn with a potential missing index. 

e The optimizer must not consider the 
query plan to be trivial. 


A trivial plan is one for which the optimizer 
determines there’s only one possible solu- 
tion. For example, if you have a query that’s 
trying to find rows in a single table with no 
indexes, no matter how many predicates 
you have, there’s only one possible plan: 
SQL Server must scan the entire table to 
find the rows you requested. So although a 
query like this might take a long time, and 
greatly benefit by indexes, the optimizer 
doesn’t store costing information for such 
a query—and it’s costing information that 
determines whether missing-index infor- 


mation is collected and stored. 

To see what I mean, create another table 
in the Adventure Works database by running 
the code below. 


USE AdventureWorks; 
GO 


IF EXISTS (SELECT 1 FROM 


sys.tables 
WHERE name = 'Orders') 
DROP TABLE Orders; 


GO 

SELECT * INTO dbo.Orders 
FROM Sales.SalesOrderDetail; 
GO 


Although the following query (Query 
1) contains a predicate on a column that 
might benefit from an index, the optimizer 
determines that the plan for this query is 
trivial, so no missing-index information is 
generated. 


SELECT * FROM orders 


WHERE SalesOrderDetailID = 40104; 


The next query, Query 2, doesn’t have a 
trivial plan, but it has no predicates, so again, 
no missing-index information is reported. 


SELECT ProductID, count(*) 
FROM orders 
GROUP BY ProductID; 


However, if you combine queries 1 and 2 
into a third query (Query 3), you get a non- 
trivial plan that includes a predicate, and so 
you'll see additional information in the sys. 
dm_db_missing_index_details view. 


SELECT ProductID, count(*) 
FROM orders 

WHERE SalesOrderDetailID = 

GROUP BY ProductID; 


40104 


Viewing an XML Query Plan 

You might be wondering how you can 
determine whether the optimizer considers 
a plan trivial. You can find the answer by 
looking at the new XML Showplan output, 


<ShowPlLanXML xmlns="http://schemas.microsoft.com/sqlserver... 


> > FIGURE | Link for XML query-plan results 
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which is available in SQL Server 2005. You 
can either use SET SHOWPLAN XML 
ON to obtain an estimated XML plan for 
a query without actually running it, or you 
can use SET STATISTICS XML ON to 
obtain an XML plan for a query as you 
execute it. For example, before running 
Query 1, I can use the following code to 
turn on the XML STATISTICS option: 


SET STATISTICS XML ON; 
SELECT * FROM orders 


WHERE SalesOrderDetailID = 40104; 


The query returns the results as usual but 
also includes a link to an XML document. 
You have to be using the SQL Server 
Management Studio (SSMS) query feature 
that displays your results in “grid” mode to 
obtain an actual link. If you're displaying 
your results in text mode, you'll see the 
entire XML document in one long string 
in the results window. I don’t recommend 
using that method. In grid mode, you'll get 
a link that looks something like Figure 1. 
When you click the link, another tabbed 
window will open, showing the entire XML 
document for the query plan with separate 
nodes for each query element. Near the top 
of the output, you should see XML that 


looks something like that in Figure 2. The 
statement-optimization level is reported as 
“TRIVIAL,” which means that no missing- 
index information will be generated. 

The XML document for the plan for 
Query 2 includes the excerpt that Figure 
3 shows. Although this plan isn’t trivial, the 
query contained no predicates, so there’s 
no missing-index information. If you look 
at the XML plan for Query 3, however, 
not only will you notice that the statement 
optimization level is “FULL,” but missing- 
index information is generated. You can see 
that information by examining the missing- 
index dynamic management views, or you 
can look more deeply into the XML plan 
document, in which you'll see the data that 
Figure 4 shows. Certain information, such 
as optimization level, isn’t available in any 
documented form other than the XML 
query-plan output. In an upcoming column, 
Pll cover the XML Showplan output in 
more detail. 


Controlling the Missing-Index 

Feature 

The missing-index feature is on by default, 

and SQL Server provides no controls or 

configuration options to adjust it. You 
cant clear the information 


StatementType="SELECT" 
Statement SubTreeCost="1.2434" 
StatementEstRows="1.12843" 
StatementOptmLevel="TRIVIAL"> 


<StmtSimple StatementText="SELECT * FROM 
Corders] WHERE ESalesOrderDetailIDJ=01" 
StatementId="1" StatementCompId="1" 


in the dynamic management 
views except by stopping and 
restarting your SQL Server 
service. No controls are pro- 
vided to turn the missing- 
index feature on or off or to 
reset any of the tables returned 
when the dynamic manage- 


> P FIGURE 2 Excerpt of XML query plan for Query 1 


ment objects are queried. 
When SQL Server is restarted, 
it loses all the missing-index 
information. 


<StmtSimple StatementText= 


StatementType="SELECT" 
Statement SubTreeCost="1.81621" 
StatementEstRows="266" 
StatementOptmLevel="FULL"> 


> > FIGURE 3 Excerpt of XML query plan for Query 2 
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"select ProductID, count(*) from 
orders&#xD;&#xA;group by ProductID" 
StatementId="1" StatementCompId="1" 


You can disable the 
missing-index feature by 
starting an instance of SQL 
Server using the -x argument. 
But the documentation for 
the Sqlservr utility (sqlserver. 
exe) doesn’t mention that -x 
affects missing-index infor- 
mation. It states only that -x 
disables the keeping of CPU 
time and cache-hit ratio sta- 
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<MissingIndexes> 
<MissingIndexGroup 
Impact="98.8402"> 
<MissingIndex Database= 
"CAdventureWorks]" 
Schema="CdboJ" 
Table="Lorders]"> 
<ColumnGroup Usage= 
"EQUALITY"> 
<Column Name= 
"ESalesOrderDetailIDI" 
ColumnId="2" /> 
</ColumnGroup> 
</MissingIndex> 
</MissingIndexGroup> 
</MissingIndexes> 


> 


FIGURE 4 Detail excerpt of XML 
query plan for Query 3 


tistics. So keep in mind that if you enable 
-x to avoid any overhead from accumulated 
missing-index information, you'll also keep 
SQL Server from gathering other perfor- 
mance-related statistics. SOL 

InstantDoc ID 95220 


SQL Server Innovators Alert! 


The 2007 SQL Server Magazine 
Innovators Contest is just around the 
corner. Starting May I, SQL Server 
Magazine will accept entries in its 
sixth annual Innovators contest. 
We're looking for solutions from 
SQL Server professionals—DBAs, 
developers, and other SQL Server- 
oriented IT folks—that demonstrate 
a resourceful, creative use of 
Microsoft SQL Server technology in 
solving an IT of business problem. 


Look for more information about the 
SQL Server Magazine Innovators 
Contest next month in the magazine 
and at http:7//www.sqimag.com. 
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Key concepts for deciding 
whether to migrate or 
rebuild your cubes in 

an Upgrade 


SQ ea 2005 Analysis Services (SSAS) provides 

many compelling reasons for upgrading from 
SSAS 2000. Among the many enhancements, SSAS 2005 
provides more flexible analysis than SSAS 2000 by exposing 
most dimension attributes as analytical objects. But executing 
on the decision to upgrade comes with some challenges. 
Should you use SQL Server 2005's built-in tools to migrate 
your SSAS 2000 cubes? Or, with all the architectural changes 
in SSAS 2005, should you rebuild the cubes from scratch? 
Both methods have pros and cons, and your decision will likely 
depend on your environment. Let’s take a look at some key 
decision points that will help you make your choice. 
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Note that this article assumes familiarity 
with basic concepts in SSAS 2005. SQL 
Server 2005 Books Online (BOL) provides 
excellent information and tutorials that 
cover these concepts. The sidebar “Upgrade 
Your Analysis Services Terminology” also 
gives a high-level guide to the SSAS 
2000 terms that have changed in SSAS 
2005. Also, to explore the pros and cons 
of the migration-versus-rebuild decision, 
we'll refer to the FoodMart 2000 sample 
database. Although FoodMart presents a 
simplistic cube implementation, many of 
the fundamental features of SSAS 2000 are 
represented in one or more of the cubes 
in this database. FoodMart also provides a 
universally understood data model, so the 
examples we use will be easy to follow. 


Migration 

You can migrate an SSAS 2000 database 
to SSAS 2005 in a few different ways, 
but all options use the Analysis Services 
Migration Wizard. For information about 
the various migration options, check out 
Darshan Singhs November 2005 article 
“Step Up to SQL Server 2005” (InstantDoc 
ID _47749). Darshan notes that before you 
start your upgrade, it’s a best practice to run 
the Upgrade Advisor to determine possible 
migration problems specific to your data- 
base. Also, you can read about overall migra- 
tion considerations in BOL (http://msdn2 
-microsoft.com/en-us/library/ms 143235 
.aspx). Known Analysis Services upgrade 
Issues include the fact that drillthrough 
options, linked objects, and remote parti- 
tions won't be migrated. 

Note that the Migration Wizard will 
make no effort to redesign your cubes to 
leverage the features of SSAS 2005. The 
wizards mission is to simply recreate the 
cube in SSAS 2005 as it was in SSAS 2000. 
This simplistic approach is by design; Micro- 
soft intended to mimimize potential prob- 


Virtually any object in SQL Server 
Analysis Services (SSAS)—including 
cubes, dimensions, data sources, and data 
source views (DSVs)—can be scripted in 
XMLA. The easiest way to begin a script 
is to right-click the object that you want 
to create in SQL Server Management 
Services (SSMS) and select the Script 
menu item. You can use this capability to 
selectively script some migrated objects. 
The following high-level checklist shows 
how to implement this method in our 
FoodMart 2000 scenario: 

1. Perform a straight database migration 
from SSAS 2000 by using the Migra- 
tion Wizard. The new database is 
probably named “FoodMart 2000.” 

2. Create a new FoodMart database, 


<Alter ObjectExpansion="0bjectProperties"” 


xmlns="http://schemas.microsoft.com/ 
analysisservices/2003/engine"> 
<Object> 
<DatabaseID>FoodMart 2005</DatabaseID> 
<CubeID>Food Mart</CubeID> 
<MdxScriptID>MdxScript</MdxScriptID> 
</Object> 
<ObjectDefinition> 
<MdxScript xmlns:xsd="http://www.w3 
-org/2001/XMLSchema" xmlns:xsi= 
"http://www.w3.org/2001/XMLSchema 
-instance"> 
<ID>MdxScript</ID> 
<Name>MdxScript</Name> 
<Commands> 
<Command> 
<Text>CALCULATE;</Text> 
</Command> 
<Command> 
<Text> 
CREATE MEMBER Sales.Measures 
.-CProfit] AS 'CMeasures] 
-[CStore Sales]-[Measures] 
-[CStore Costi FORMAT_ 
STRING = 'Standard'; 
</Text> 
</Command> 
<Command> 
<Text> 
CREATE MEMBER Sales.Measures 
.[Sales Average] AS 
'[Measures].[CStore SalesJ/ 
CMeasures].[Sales Count]', 
FORMAT_STRING = 'Standard'; 
</Text> 
</Command> 
</Commands> 
</MdxScript> 
</ObjectDefinition> 


</Alter> 


named something like “FoodMart 2005.” 


. Script and run the data source from the migrated database (change the database 


in the script). 


. Script and run the DSV from the migrated database (change the database in 


the script). 


. Add the appropriate relationships and primary keys to the DSV in FoodMart 


2005. 


. Rename named calculations in the FoodMart 2005 DSV as appropriate. The 


Migration Wizard added some named calculations to some of the tables in the 
DSV and named them “Column1.” These named calculations reflect SQL state- 
ments that were entered into the Source Column for cube measures or Member 
Column for dimension-level names in FoodMart 2000. It’s a good idea to rename 
these to better represent the underlying calculation. 


. Create a new cube in FoodMart 2005 by using the Auto Build feature. 
. Review and revise the resulting dimensions (e.g., attribute properties, user 


hierarchies, attribute relationships) in FoodMart 2005. 


. Script and run the XMLA to create other objects, such as partitions, MDX 


calculations, and actions to copy them from the migrated database to FoodMart 
2005. You can script these objects independently or extract them from a cube 
or database creation script. 


10. Add designations for parent/child, custom member formulas/rollups, and 


unary operators to FoodMart 2005. You can determine these designations by 
reviewing the migrated database. 


lems with front-end applications. Knowing 
about this design will help you more easily 
understand the choices that the wizard 
makes. 

You can migrate the FoodMart 2000 
sample cube by opening an SSAS 2005 
instance in SQL Server Management Studio 
(SSMS) and walking through the Migration 


11. Add drillthrough actions to duplicate drillthrough settings from SSAS 2000. 


It might be easiest to cut out subsets of a cube or database creation script when 
creating objects such as partitions, actions, and MDX calculations. You can run the 
example script in Listing A in SSMS to create calculated members from the source 
Sales cube in the newly generated database. 

InstantDoc ID 95163 
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Wizard (right-click the instance name and 
select Migrate Database). The wizard will 
ask for predictable information regarding 
the source SSAS 2000 server and the data- 
bases you want to migrate. After running the 

Migration Wizard, you can open Business 

Intelligence Development Studio (BIDS) 

and view the resulting SSAS cubes and 

shared dimensions, which Figure 1 shows. 

Theres some room for interpreta- 
tion about how FoodMart should be 
implemented on SSAS 2005, but most 
people agree that the migrated database 
in Figure 1 isn’t the best design. The fol- 
lowing documents some of the high-level 
observations: 

e Because of SSAS 2005's ability to cre- 
ate more comprehensive cubes than 
SSAS 2000, the four regular cubes 
should be collapsed into one. It can be 
argued that the HR cube could stand 
on its own. 


Upgrade Your 
Analysis Services 
Terminology 


The new terminology in SQL Server 
2005 Analysis Services (SSAS) can 
seem intimidating but can be simpli- 
fied by the equivalency chart that 
Table A shows. Because SSAS 2005 
is a whole new paradigm, it can be 
argued that these terms aren’t exact 
equivalents, but this list can help you 
get enough of a grasp on the new 
concepts to get started. 


e Virtual cubes in SSAS 2000 were 
migrated as cubes because no equiva- 
lent for a virtual cube exists in SSAS 
2005. We shouldn’t need virtual cubes 
when the SSAS 2000 cubes are con- 
solidated into a single SSAS 2005 cube. 

e Several of the 


** Analysis Services Migration Wizard 


Completing the Wizard 


The migration is complete. You can review the new database structures. 


The following databases have been migrated to EWELKER\SQL2K5. 


Preview: 


=| (& Cubes: 
D Budget 
B HR 
J Sales 
(@ Warehouse 
(@ Trained Cube 
| Warehouse and Sales 
=| |S} Dimensions: 
LZ Account 
LZ Category 
LZ Customer Pattem 
LZ Customers 
LZ Department 
LZ Education Level 
LZ Employees 
TZ Gender 
LZ Marital Status 
LZ Position 
LZ Product 
LZ Promotion Media 
LZ Promotions 
Z Store 
UZ Time 
Vay Warehouse 
val Yearly Income 
LZ Years in the Company 
IZ Pay Type 


dimensions we 
got are varia- 

A tions on what 
should be a single 
dimension in 
SSAS 2005. For 
example, Gender, 
Education Level, 
Marital Status, and 
Yearly Income 
should become 
attribute hierar- 
chies in a single 
Customer dimen- 
sion. Note that 


> > FIGURE | Cubes and shared dimensions of FoodMart 


cube after migration 
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TABLE A Terms That Have Changed 
from SSAS 2000 to SSAS 2005 


Analysis Services 2000 Analysis Services 2005 
Cube Measure group 
Virtual cube Cube 


Virtual dimension Attribute dimension 


(single level) 


Virtual dimension 
(multiple level) 


User-defined hierarchy 


Drillthrough properties Drillthrough action 


Member property Attribute 


InstantDoc ID 95162 


these attributes actually appear in the 
Customer dimension because they 
were defined as member properties in 
SSAS 2000. 


Another issue is the ability to easily add 
new attribute dimensions (which are a cor- 
nerstone of SSAS 2005) to any of the core 
dimensions. The data source view (DSV) 
that the Migration Wizard creates includes 
only columns that support the migrated 
objects. The Customer table in the source 
FoodMart 2000 relational database contains 
several columns that might make for rich 
analysis options, such as city, state, customer 
region, total children, houseowner, number 
of cars owned, and more. You can update the 
DSV to include all columns from the source 
tables by right-clicking the background in 
the DSV designer and selecting Refresh. But 
do this at your own risk; some properties of 
pre-existing attributes might change. You 
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might find it easier to simply replace specific 
dimension tables in the DSV (right-click 
the table, select Replace Table, With Other 
Table, then select the same table). Once 
you've updated the DSV with all available 
columns, you'll need to edit each dimension 
to add desired attribute hierarchies from the 
newly available columns. 

Despite the negatives, a lot of good stuff 
happened during this migration. All of the 
MDX calculations that we defined in calcu- 
lated members migrated successfully (look 
under the Calculations tab for the Sales 
cube in BIDS). Our parent/child dimen- 
sions (Account and Employees) migrated 
successfully. And the partitions and aggrega- 
tion designs, actions, unary operators, and 
member properties were retained and are 
operational. Finally, note that the Migra- 
tion Wizard will migrate only metadata. 
Migrated cubes must be fully processed to 
populate the cubes with data. 


Starting from Scratch 

Building a cube from scratch isn’t as extreme 
as it first sounds. The main thing you 
need is a well-designed star or snowflake 
schema. You start in BIDS by adding a new 
data source, then a new DSV. Create the 
new DSV in the BIDS Solution Explorer 
by right-clicking Data Source Views and 
selecting New Data Source View. The 
wizard will guide you through selecting the 
data source and adding the tables required 
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for all cubes (which will be measure groups 
in SSAS 2005). 

When you view the new DSV in the 
DSV designer, you'll note several missing 
primary key and foreign key relationships. 
Although adding these relationships isn’t 
required, doing so can help SSAS form 
better queries against the data source when 
refreshing dimensions or cubes. Also, if you 
don’t add the appropriate relationships in the 
DSV and you use the Auto Build option to 
invoke Intellicube to build the cube, Intel- 
licube wont be able to differentiate fact 
tables from dimen- 
sion tables. In the case 
of FoodMart 2000, 
you'll notice that the 
five tables (account, |— - 
category, department, 
product_class, and 


® Cube Wizard 


Time dimension table: 


=, 
| Tables | Diagram | 


and examining the underlying relationships. 
You can modify relationships in the DSV 
designer. Note that these added relation- 
ships are in the DSV only. The DSV is an 
abstraction of the source and doesn’t result 
in changes to the source. 

After creating the appropriate relation- 
ships, you can create a new cube in SSAS 
2005 by right-clicking Cubes in the Solu- 
tion Explorer in BIDS, then selecting New 
Cube. Make sure Auto Build is selected to 
invoke Intellicube. The wizard now lists 
only one dimension table incorrectly as a 


Identify Fact and Dimension Tables 
Identify Fact and dimension tables in the data source view, You can also 
specify a time dimension table. 


<None> 


product) show up 
as fact tables if you 
attempt to auto-build 
the cube without 
creating these rela- 
tionships. They show 
up this way because 


| © dbo.account 


IS dbo.store 
(Qi dbo.employee 


{@ dbo.position 
[@ dbo.salary 
| [Ì9 _dbo.time_by_day 
Lita dha nramotian. 


dbo.inventory_fact_] 


1997 


JESOOs8oO 
THE HSOOSH 


v 


they have no defined 
relationships to other 
tables in the DSV. If 
you uncheck the box 
for the table in the 
Fact column, yov’ll 
see the warning mes- 
sage that Figure 2 
shows. So, its impor- 
tant to create relation- 


© Cube Wizard 


necessary. 


Review New Dimensions 
Review the structure of the new dimensions and change the structure if 


A One or more tables may have been incorrectly selected. 
The Following dimension tables are not referenced by a Fact table: dbo_account 
> > FIGURE 2 Warning to identify fact and dimension tables 


ships between the fact New dimensions: 
and dimension tables. E ia Category 
A w Iv Time By Day 
Another good E [F Lat Account 


Œ [VIS Store 
E [V IE Product 
E [VIE Warehouse 


practice is to add log- 
ical primary keys to 
all dimension tables. 
You can create rela- 
tionships based on 
your knowledge of 
the underlying data 
source. If the data 
source isnt available, 


E [VIE Customer 


you can easily view 
it by bringing up the 


cube editors in the 
SSAS 2000 databases 
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> > FIGURE 3 New dimensions that the Cube Wizard produces 
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TABLE | Desired Effects of a FoodMart 2000 Migrati sone Le tae al 
esired Effects of a FoodMart igration 2005. The biggest down- 
Desired Effect Migration Rebuild side to starting from 
© oe 393 
Dimension hierarchies that y scratch is that the “extras 
share source dimension that existed in the SSAS 
taltes emcee 2000 cube aren’t reflected 
dimension. . 
in our brand-new cube. 
Cubes that share dimen- d Namely, we have to 
y, 
sions result in a single manually set up calcu- 
cube with multiple measure lated b / 
groups. ae eee ers, parent 
A z child dimensions, actions, 
Multiple dimensions result in V b II 
a single dimension. custom member ro ups, 
and una: operators Just 
Calculated members are V h £y OP T i 
retained and work. as t RA were initially 
: ; set up in SSAS 2000. 
Virtual cubes disappear; V If atendi b 
a single cube is all that is WED a o use cube 
necessary. partitioning, we'll have 
Parent/child dimensions are V to re-implement this 
retained as parent/child. functionality in the new 
Virtual dimensions become Sometimes y database. 
part of the dependent We also need to deter- 
gimena mine aggregation design 
Private dimensions become V before processing the 
part of the root dimension. cubes. Before designing 
Multiple partitions are V aggregations, be sure to 
EMR endilega ega evaluate the dimension 
tion design. x 
$ attributes to be used and 
Aetions migrate: V set AttributeHierarchy- 
Drillthrough options migrate. Enabled to False on those 
* EJ 
E j attributes that won?t be 
and custom rollups migrate. used for analysis. 
Unary operators migrate. V Š 
So, What’s the 
Member properties migrate. V y 


fact table, presumably because of its rela- 
tionship to other tables (i.e., itself ).You can 
simply uncheck the box in the Fact column 
next to the Employee table to remove this 
designation. 

The Cube Wizard produces 10 dimen- 
sions, versus the 19 dimensions that the 
Migration Wizard created. These 10 dimen- 
sions, which Figure 3 shows, are more 
“pure”—all related attributes, hierarchies, 
and properties are contained within a single 
dimension. And the Cube Wizard has cre- 
ated just one cube. (We could have elimi- 
nated the HR information by removing it 
in the DSV or not selecting it during the 
cube creation). 

As you can see in Figure 3, the resulting 
dimensions and cubes reflect the desired 
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Best Plan? 


In upgrading SSAS 
cubes, both migrating and rebuilding cubes 
have pros and cons. You might want to 
initially perform a straight migra- 
tion just to take advantage of 
new engine features such as 
more efficient memory 
management and multi- 
instance supportin SSAS 
2005. Even when re- 
architecting the cubes, 
you might find it benefi- 
cial to ultimately leverage 
both methods. Table 1 gives 
a high-level overview of the 
benefits of both methods. 

As you can see, the ben- 
efits of the two methods 
have very little overlap, which 
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makes combining the two methods even 
more appealing. You can migrate the data- 
base by using the Migration Wizard, create 
a new database and build the cube from 
scratch, script objects in the migrated 
database, and run the scripts in the new 
database as a means of copying the desired 
objects from the migrated cubes. The sidebar 
“Scripting a Cube in XMLA,” page 34, 
describes how you can create a script to run 
in SSMS to create calculated members from 
the source Sales cube in the newly gener- 
ated database. 

Although I’ve tried to address the most 
common considerations for migrating cubes 
from SSAS 2000 to SSAS 2005, I don’t have 
enough space here to include every possi- 
bility. For example, neither of the methods 
I discussed optimizes aggregation usage and 
attribute relationships. But you can use the 
ideas in this article to help you search BOL 
for specific migration issues that you might 
encounter. SOL 
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Rapid access to real-time metrics 


M any factors affect the performance of 
your Microsoft SQL Server—based 
applications. When application performance 
begins to suffer, determining root causes 
can be challenging. This is when real-time 
performance monitoring and management 
tools can make a difference. I’ve reviewed 
two easy-to-implement solutions that will 
make your life easier and help you reach 
resolution faster when your SQL Server 
solutions go south: Idera’s SQL diagnostic 
manager and Quest Software’s Spotlight 
on SQL Server Enterprise. Both products 
monitor SQL Server and Windows OS 
metrics and generate alarms when pre- 
defined threshold values are crossed. Both 
display server status graphically and let you 
drill down into more detailed information. 
Both let you archive metrics to a SQL 
Server database and provide sample reports 
in SQL Server 2005 Reporting Services 
(SSRS) format. 

I had intended to include BMC Perfor- 
mance Manager for Databases in this review. 
Unfortunately, after spending more than two 
days obtaining and attempting to figure out 
how to successfully install the product, I ran 
out of time. To be fair, I’m sure the suite 
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of products that BMC sells is top-notch, 
once you get them installed and working. 
But the difficulty I experienced using the 
documentation to install the product forced 
me to eliminate it from consideration in this 
review. 


SQL DIAGNOSTIC MANAGER 
Idera’s SQL diagnostic manager (SQLdm— 
I reviewed version 4.6.8) is an easy-to-use 
tool to manage and monitor multiple 
instances of SQL Server 2005/2000/7 in 
real time across an enterprise. SQLdm is a 
full 24 x 7 monitoring solution with drill- 
down capability for problem analysis and 
archiving for historical analysis. Whereas 
other companies offer performance-moni- 
toring products to support different vendors! 
solutions, Idera focuses solely on products 
for SQL Server. 


Architecture 

SQLdm has a fairly simple design. Idera 
recommends that you install it on a server 
that doesn’t host SQL Server. A SQLdm 
service queries the SQL servers that you tell 
it to monitor. SQLdm installs nothing on 
the monitored server—no agents, services, 
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databases, or stored procedures. SQLdm 
uses native SQL Server monitoring inter- 
faces—DMO (Distributed Management 
Objects), SMO (Server Management Ob- 
jects), remote queries, and (when turned 
on) a remote lightweight trace—to obtain 
information from monitored servers. During 
quiet time that you configure, SQLdm 
queries for database growth and table reor- 
ganization information and also queries 
Performance Monitor counters for OS 
metrics. By default, data is stored in flat files 
on the SQLdm server for seven days. With 
the SQLdm Metrics Repository, you con- 
solidate and move the data to a SQL Server 
database for historical analysis and after-the- 
fact problem determination. A suite of cus- 
tomizable predefined reports that use SSRS 
queries the Metrics Repository and lets you 
create a user-driven reporting system. Web 
Admin, an independent module that is part 
of the SQLdm package, remotely monitors 
SQL servers and offers a limited subset of 
the metrics and monitoring capabilities of 
SQLdm’ server-based GUI. 


Installation 
SQLdm would be a snap to install if Idera 
would make the installation guide a little 
more accessible and state system prerequi- 
sites a little more clearly. After I downloaded 
the installation executable, I wanted to 
download an installation guide. I looked 
around Idera’s Web site and couldn’t find 
one, so I sumply forged ahead. As it turned 
out, the installation package presented an 
option to view a one-page installation sum- 
mary as soon as it finished extracting files. 
SQLdm has two installable components:The 
first installs core data collection components 
and a GUI console. The second installs the 
Metrics Repository, which lets you archive 
data for reporting and trend analysis. Web 
Console is a third, independent component 
that monitors only SQL Server 2005 and 
SQL Server 2000 databases and exposes a 
subset of the metrics available in SQLdm. 
Installation requirements are basic: a 
1.1GHz Pentium or better processor, 384MB 
of RAM, Microsoft Data Access Compo- 
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> > FIGURE | SQL Diagnostic Manager presents an effective summary 
of monitored servers and alarms 


nents (MDAC) 2.8, Microsoft .NET Frame- 
work 1.1 SP1, SQL Client Tools, and Internet 
Explorer (IE). For some reason, the installa- 
tion summary doesn’t mention the need for 
SQL Client Tools, but the installation routine 
prompts you for it later if it’s not installed. 
I installed Connectivity Components and 
Management Tools from the Client Com- 
ponents section of SQL Server 2005 Setup, 
and this made the primary SQLdm setup 
routine happy. As I discovered later when 
many of the metrics I expected weren't avail- 
able, SQLdm requires the SQL Server 2000 
Client Connectivity tools—the SQL Server 
2005 version I installed wasn’t sufficient. 
The downloadable installation program 
presents a simple yet effective Web-based 
Getting Started interface. Installation of both 
SQLdm and the Metrics Repository took 
just a few minutes after I configured the 
server with the requisite software. Installing 
the Metrics Repository database required a 
target SQL Server instance for data storage. 
You use the SQLdm Metrics Repository 
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PROS: Strong drill-down capabilities; effective 
console display 


CONS: Weak Web console for remote access; 
weak multiple-console support 


RATING: KKK 


PRICE: $1295 per monitored SQL Server 
instance 


RECOMMENDATION: Product of choice for 
shops without significant need for multiple- 
console or remote access. 


CONTACT: Idera © http://www.idera.com 


Service Manager to configure how often to 
write monitored metrics to the repository. 
Both SQLdm and the Metrics Repository 
run as services. Web Console installs quickly 
on a system configured with Microsoft 
IIS and ASPNET and doesn’t need to be 
installed on the same system with SQLdm. 


The GUI Console 

SQLdm’s GUI makes it easy to add moni- 
tored servers, view the current status of each, 
and drill down into the views for extended 
analysis.As Figure 1 shows, on the left side of 
the GUI you'll find a tree view of all moni- 
tored servers, along with an All Servers view. 
The All Servers view consolidates selected 
information across all monitored servers, 
allowing you to view and filter key kinds 
of information. 

The Summary display panel of the GUI 
graphically displays the status of each moni- 
tored server. Icons display the status of key 
services, and small graphs show CPU utiliza- 
tion, user counts, I/O rates, and SQL batch 
execution over the past hour. Right-click 
menus offer access to a wealth of detailed 
metrics, and context-sensitive Help is avail- 
able. For example, positioning the cursor 
over a point on a graph displays the metric 
for that area. Clicking in a servers graphical 
display brings up the Server Summary screen, 
which includes key metrics and the status of 
each database. The SQL Performance and 
OS Performance tabs in the Server Sum- 
mary screen graphically display recent values 
for key metrics for the server. The Details 
view, an alternative to the Summary view, 
displays the status of monitored servers. Sites 
with large numbers of SQL servers will use 
the Custom View feature: By assigning each 
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server a server group, application group, 
department, and location, you can create a 
variety of custom views that are easily select- 
able from a drop-down box. 

To Do List is the third window in the GUI. 
Here SQLdm displays alerts when a moni- 
tored metric exceeds one of the predefined 
threshold values. Double-clicking an entry 
lets you drill down into additional informa- 
tion relating to the event—a particularly easy 
way to start digging for root causes. Clicking a 
check box marks an event complete. 

After installing the three components of 
SQLdm, I used the primary GUI to begin 
monitoring several servers. Using either the 
wizard or the direct-entry methods was 
quick and easy, and in each case SQLdm 
immediately displayed graphical evidence of 
monitoring. Since SQLdm installs nothing 
on monitored servers, there was no delay in 
displaying results. 

As I exercised the product further, I 
discovered that I needed to enable OLE 
Automation on monitored SQL Server 
2005 instances in order to collect the met- 
rics displayed on the OS Performance tab of 
the Server Overview screen. Although OLE 
Automation is something one normally 
enables by using the Surface Area Configu- 
ration for Features tool in SQL Server 2005, 
Idera made enabling OLE Automation a 
one-click procedure from the OS Perfor- 
mance tab. 

Monitoring queries and stored proce- 
dures were the only other functions that I 
needed to enable for each monitored server. 
SQLdm reports the worst performing and 
the most frequently run stored procedures, 
triggers, single statement SQL queries, and 
SQL batch queries filtered by the query 
types and performance thresholds you 
configure. 

An Idera representative told me that with 
full monitoring enabled, server overhead 
typically doesn’t exceed the 2 to 4 percent 
range. Although I wasn’t able to validate this 
claim, I saw nothing that would lead me to 
doubt it. 

SQLdm will also collect table growth 
and fragmentation statistics during a daily 
two-hour quiet time window that you 
specify on selected days of the week, for 
selected databases. 
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The drill-down tools are a powerful way 
to not only see real-time metrics but also 
to select (by day and date range) and view 
historical raw metric data (with a default of 
one week) for after-the-fact analysis. SQLdm 
lets you export displayed graphs or data to the 
Clipboard or Excel with a mouse click. 

SQLdm offers a flexible alerting system 
that is configurable at a granular level. For 
monitored metrics, you can set warning and 
error threshold levels for each monitored SQL 
instance. Notification destinations are con- 
figurable by SQL Server instance, metric, and 
severity level. SQLdm supports several notifi- 
cation destinations: to a list of email addresses, 
Windows application event logs, Windows 
messages, any ODBC data source, and an 
external script. For jobs, SQLdm lets you 
filter alerts by SQL job category and lets you 
configure alerts on abnormally ending jobs 
and on jobs that run a specified percentage of 
tume longer than the average run time. 


Web Console 

The Web Console takes some configuring 
in the beginning: You must configure servers 
and server groups, and the accounts that will 
authenticate access to them. One limitation 
is that all users are equal. Although you can 
create server groups to make it easy to navi- 
gate to the servers you manage, all users have 
the same access to all servers. The interface 
is reminiscent of Outlook 2003, with three 
primary sections: Server Status, Agent Status, 
and Performance Analysis. The information 
presented is a small subset of what’s available 
by using the GUI, and you are limited to 
viewing one server at a time—there is no 
“all servers” consolidation that the console 
GUI includes. 

The lack of a way to install multiple 
consoles that connect to and share a single 
data collection service is a limitation that 
Idera plans to eliminate in the next SQLdm 
release, due out later this year. 


SPOTLIGHT ON SQL SERVER 
ENTERPRISE 

Quest Software’s Spotlight on SQL Server 
Enterprise is another easy-to-install, quickly 
usable product for monitoring instances of 
SQL Server 2005 and 2000. Michael Camp- 
bell reviewed Spotlight on SQL Server 5.7 
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(not the Enterprise version) in February 
2007 (InstantDoc ID 94548). PI highlight 
the additional features of the Enterprise 
version and expand on the two products’ 
common features. 


Architecture 

Key differences between the standard and 
Enterprise versions of Spotlight on SQL 
Server are architectural. In the standard edition, 
each Spotlight installation connects directly to 
the monitored SQL servers. When monitoring 
a SQL Server instance from several locations 
Ge., from several installations of Spotlight), 
each location will connect to the monitored 
server independently. The Enterprise edition 
introduces a Diagnostic Server and an Enter- 
prise Viewer. The Diagnostic Server connects 
to each monitored server and manages data 
collection. Multiple GUI consoles can con- 
nect to the Diagnostic Server, ensuring that 
each monitoring location has the same view 
of monitored SQL Server instances while 
eliminating the load that additional, redundant 
connections would impose on the monitored 
server. The Diagnostic Server runs as a service 
and incorporates its own database system for 
short-term data storage. When you create a 
connection to a SQL Server instance, Spot- 
light installs a database and stored procedures 
on that monitored instance that it uses for data 
collection. Spotlight on Windows Enterprise 
is included in both versions of Spotlight on 
SQL Server and adds the ability to monitor 
and archive Windows OS metrics. In the 


SPOTLIGHT ON SQL SERVER 
ENTERPRISE 
PROS: Good multiple-console support; console 


easily configurable whether you have few or 
many SQL servers 


CONS: The console would benefit from some 
usability tweaks; reporting is relatively weak, 
with few templates 


RATING: WWW 
PRICE: $1995 per monitored server 


RECOMMENDATION: The product of choice 
for the largest shops. 


CONTACT: Quest Software © http://www.quest 
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GUI you'll see two connection icons for each 
server when Spotlight is monitoring both 
SQL Server and Windows metrics. 

When you ask Spotlight to monitor a 
SQL server, it installs a work database with 
stored procedures to assist data collection on 
each monitored instance and doesn’t install 
an agent service to run on the system. The 
Enterprise edition looks at lots of data on 
the server: Performance Monitor, system 
tables, Data Management Views in SQL 
Server 2005 using native SQL Server, and 
OS instrumentation. 

The Diagnostic Server that is part of 
Spotlight on SQL Server Enterprise is 
responsible for data collection and for com- 
munication to the instances of the console 
program that might be installed around the 
enterprise. The Diagnostic Server keeps the 
data it retrieves for one week, allowing you 
that much time to review detailed metrics 
in support of root cause analysis. Another 
component, installed separately but included 
with the basic license, is the Spotlight 
Reporting Option. In addition to reporting, 
the Reporting Option also lets Spotlight on 
SQL Server Enterprise users archive data 
to a SQL Server database for long-term 
trend analysis. Using the Reporting Option 
Configuration tool, you can easily create 
the database and enable automatic data 
collection. As for producing reports, Quest 
provides three sample reports for use with 
SSRS to get you started. 


Installation 

How many ways can you say easy? Spotlight 
on SQL Server Enterprise requires Windows 
Server 2003, XP, or Win2K with MDAC 2.8 
and SQL Server 2000 Client Tools. Storage 
space on an instance of SQL Server is needed 
to use the Reporting Option. The Spotlight 
on SQL Server Enterprise Getting Started 
Guide outlined everything I needed to know. 
Upon starting up the console after running 
the installation routine, I was prompted to 
either connect to an existing Diagnostic 
Server or install a new one. I selected Install 
and the resulting wizard completed quickly. 
Just as easily, I used Spotlight on SQL Server 
Enterprise’s discovery process to find and 
configure SQL servers for monitoring. For 
kicks, I reran the Diagnostic Server installa- 
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> > FIGURE 2 Color and motion highlight areas of interest on Spotlight 


for SQL Server Enterprise’s single-server displays 


tion wizard, directing it to install the service 
on a remote server, and the process was just 
as clean. The documentation warns that error 
log scanning can affect the performance of 
the monitored server, and Spotlight on SQL 
Server lets you configure what to scan for in 
the error log. 

The Reporting Option installs on the 
Diagnostic Server from a separate down- 
loadable file. Once the Reporting Option 
is installed, you use the Spotlight Reporting 
Option Configuration tool to specify a target 
SQL Server instance for storage and, if neces- 
sary, create the database. A Configuration tab 
lets you select which classes of metrics will be 
archived and how often. A Connection tab 
lets you override the global defaults of the 
Configuration tab for specific connections. 


The Spotlight Console 

The Spotlight console offers several views of 
the activity of your monitored servers. The 
Connection view, which Figure 2 shows, 
displays the status of a single SQL server. The 
first thing you notice is the bright, active 
presentation. Indicators spin and flow to 
represent rates of activity and flash to draw 
your attention to metrics passing an alert 
threshold. The Spotlight Enterprise Today 
perspective lists current alarm conditions 
across all monitored connections. 

Shops with a larger number of SQL 
servers will appreciate the Enterprise Viewer, 
which is simply another selectable view 
within the console. Enterprise Viewer pres- 
ents you with a configurable high-level view 
of the status of your systems. Monitored 
servers appear as simple labeled circles that 
are color coded for the highest severity alarm 


currently raised on the server. You can create 
groups and subgroups of servers that display 
as a single icon, allowing you to represent the 
status of the largest enterprise on one screen. 
The Enterprise Viewer also let me create and 
save multiple named views, each with a dif- 
ferent subset and grouping of servers. 

Overall, I felt that the multiple entry points 
for the various views that Spotlight on SQL 
Server Enterprise can present detracted from 
its overall usability. I can see a need to hide 
inactive connections in a situation where you 
have a large number of monitored servers, 
but why not provide a simple check box 
to highlight open connections when all are 
displayed? The Enterprise Viewer could also 
be integrated more tightly into the console. 
The Enterprise Today view, which 1s really 
a filtered display of alarms, is a useful display, 
yet its hidden in the “Welcome” group 
with Web links that are essentially gratuitous 
marketing, and displaying it hides your list of 
views or connections, adding an extra step 
to switch back out of. Separating OS- and 
SQL-related displays into two connections 
also clutters the selection column, especially 
for those with lots of servers. 

No matter which view you're looking at, 
Spotlight on SQL Server Enterprise lets you 
know when a high-severity alarm is raised by 
flashing an iridescent red icon that includes 
the SQL Server instance. When Spotlight 
on SQL Server Enterprise is minimized or 
not the active focus of the monitor, a balloon 
pops up from the Task Bar notification icon 
with the alarm summary. To drill down into 
the alarm, simply expand the group until you 
get to the server raising the alarm, then display 
the home page for the flashing connection. 
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Positioning the cursor over the servers pri- 
mary status icon displays the alarm summary. 
From there you can quickly drill down into 
the various detail status pages that the alarm 
indicates to explore the root cause. 

I asked Quest Software what overhead 
Spotlight imposes on a monitored server, 
and the answer was, “It depends.” I was told 
that when a Spotlight console is connected 
to and actively monitoring a server, the 
overhead is in the range of 4 to 4.5 per- 
cent. The console calls for more frequent 
sampling than does the Diagnostic Server 
alone, which causes less than 1 percent CPU 
overhead. 

A key benefit to Spotlight on SQL 
Server Enterprise is the ability to install the 
console at additional locations, each con- 
necting to the same Diagnostic Server, so I 
installed a second Spotlight on SQL Server 
Enterprise console on an XP system. The 
installation was easy, and I pointed it to the 
Diagnostic Server I had set up previously. 

Spotlight has a lot of flexibility as you 
define alarms. It allows you to define as 
many as seven severity levels of alarms for 
monitored metrics. Alerting/alarm reporting 
options are relatively limited: For each 
metric, email is the only external reporting 
option, and you are allowed to enter a 
single email address for notification. I liked 
Spotlight’s ability to send an email message 
when it detects a “server down” condition. 


Summary 
Both of these products are usable and effec- 
tive. Many people will like SQLdm’s price 
point and features, and I found it to be a bit 
easier to use than Spotlight on SQL Server 
Enterprise. SQLdm does have its limitations, 
notably in the area of remote monitoring 
and multiple-console support. Spotlight has 
the edge here with its Console/Diagnostic 
Server architecture. Combined with the 
ability to create very high-level summary 
views of the health of the largest database 
farm, the largest installations may well look 
to Quest. It was a difficult decision, but my 
Editor’s Choice for this review goes to SQL 
diagnostic manager for its usability features 
and likely appeal to a very broad audience. 
E 
InstantDoc ID 95281 


www.sqimag.com SQL Server Magazine Apil2007 4l 


Hear from Gartner Analyst John Enck about the latest trends in 

Windows and the *Nix platforms. Join industry experts Michael New York 
Otey, Darren Mar-Elia and Brian Komar for practical tips to man- 

age and secure your heterogenous environment. May 1 


TechX has something for everyone, regardless of whether you Washington, DC 
consider yourself a Windows person or a Linux/UNIX diehard. It's May 3 
the one event where the two camps can come together to find 


common- ground; San Francisco 
; ing h 1 will learn at Tech May 8 
| Where to find and how to use new tools to help Ki Overview of today’s virtualization tech- 
you monitor your heterogenous environment nologies from hardware to applications and y 
Ki How to manage and secure your non-Window practical tips for using virtualization 
systems using Group Policy Ki How virtualization can help you solve com- 
Ki How to break down the past communication bar- mon IT challenges such as server sprawl, 
riers between Windows and *Nix systems deployment and testing 


[Z] How to secure and manage data and access 
management across heterogenous environments 


Hurry, seats are filling fast! Register today 
at www.windowsitpro.com/go/techx2007 


Windows IT Pro and Microsoft are bringing you a live, 
under-the-hood look at Longhorn this spring, featuring 
virtualization, web services and core reliability 
breakthroughs. You will get in-the-trenches insights 
from the smartest people in the business, and you 

will walk away knowing what new features and 
enhancements are included. Plus, you will receive 
practical advice on how you can maximize Longhorn 

in your environment. 


| Spaceis s limited, s so register today at www.windowsitpro.com/go/longhorn 


bse ean 


Got a great new product? 
Send announcements to products@sqlmag.com. 


New Products 


BACKUP AND RESTORE 


Backup and Restore for Shared 
SOL Server Hosting 


myLittleTools announced myLittleBackup for SQL Server 2000 and 2005, a backup and 
restore Web application for shared SQL Server hosting. myLittleBackup lets users upload 
a backup file from their own server and restore (install) it on the shared SQL server. 
After the restore, myLittleBackup automatically checks user mappings and other options. 


Blake Eno (products@windowsitpro.com) is product editor for Windows IT Pro and SQL Server Magazine. 


The solution includes two 
tools that help reduce sup- 
port time. The “Check data- 
bases” tool lets users fix 
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Restore a Database to 
a Point in Time or Point 
of Failure 


Databk.com announced SQL Server 
Backup 6.1.2, a backup and restore utility 
that uses data compression and 128- or 
256-bit encryption to perform full database 
backups, differential backups, and transac- 
tion log backups. SQL Server Backup sup- 
ports multiple SQL Server instances and 
can restore a database to a point of failure 
or specific point in time. The software also 
deletes expired backups automatically and 
can email reports to inform you about the 
status of a backup or restore job. SQL Server 
Backup supports Microsoft SQL Server 
2005/2000/7.0 and SQL Server 2005 
Express. SQL Server Backup is priced at 
$79.95, and volume discounts are available. 


user mappings and other 
options without opening a 
support ticket. The “Manage 
backup files” tool lets users 
download, upload, and 
delete old backup files. For 
more information, contact 
myLittleTools at support@ 
mylittletools.net, sales@ 
“mylittletools.net, or http:/7 
“www.mylittlebackup.com. 


You can download a free 15-day trial ver- 
sion of the software at Databk.com’s Web 
site. For more information, contact Databk 
.com at support@databk.com, sales@databk 


.com, or http://www.databk.com. 


DATABASE DOCUMENTATION 


about your databases and can add or 
amend object descriptions and display all 
cross-database dependencies. The software 
exports the documentation in HTML with 
or without frames. SQL Doc Pro edition 
includes a command-line interface. A free 
14-day trial of SQL Doc is available at Red 
Gate’s Web site. SQL Doc starts at $295 
for the Standard edition and $495 for the 
Pro edition. For more information, contact 
Red Gate Software at http://www.red-gate 
com. 


PERFORMANCE 


Defragment SQL Server 
Databases 


Idera announced SQL defrag manager, 
an automated analysis and defragmenta- 
tion solution for SQL Server databases that 
improves SQL Server database performance 
by continuously analyzing fragmentation 
levels and automatically defragmenting 
indexes. SQL defrag manager’s management 
console provides a real-time window into 
fragmentation levels and lets you manage 
and control automated analysis and defrag- 
mentation activity across multiple servers 
and databases. SQL defrag manager also 
determines defragmentation scheduling to 
ensure that critical production servers will 
be minimally affected. The software is priced 
at $995 per SQL Server instance. For more 
information, contact Idera at 713-523-4433, 


877-464-3372, or http://www.idera.com. 


Simplify Database 
Documentation ae ida 


Red Gate Software announced 
SQL Doc, a tool for documenting 


SQL Server 2005/2000 data- | -<) =~ 
bases. SQL Doc lets you access vce 
database overviews—including Eu ; 
schema and dependencies—and Ss . 
select which databases or sec- ae 
tions of databases you want to Be 
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document, down to the object |p= 
level. SQL Doc uses Red Gate 


APIs to retrieve information 
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Implement, Maintain, and  Enenra Pn 
mpiement, Maintain, an A = 
2005 Databases GoldenGate Software announced GoldenGate 9.0, transactional data management 


software that improves the availability, accessibility, and performance of mission-critical 
data. The software reads the -— 
SQL Server database log, 1 TATS =m ae 
encrypts committed transac- 
tions, and moves them across 
the network to one or many B a 

targeted systems. To ensure . E- TE- 9 Je o = 
continuous uptime, Golden- l X A 

Gate moves the data bi-direc- =a 
tionally while maintaining 


KeyStone Learning Systems released its 
latest instructor-based video courseware, 
“SQL Server 2005 Implementation and 
Maintenance.’ This training course starts 
with the processes of installing and con- 
figuring SQL Server 2005 and upgrading 
SQL Server 2000 to SQL Server 2005. 
The training then takes you through sup- 
porting database clients, importing and 
exporting data, and working with XML ’ aad eer die 
data. “SQL Server 2005 Implementation TA ETE ee 
: 55 f a primary system outage, == Se 

and Maintenance” explains how to create : —— Ss Ss homens Capes eee ee 

: . users are pointed to a second S mamen wei E Sms Sa mma a IGSS i N A 
and manage various database objects, 
: ; . : , fully synchronized system. 
including tables, views, triggers, functions, 3 ; 

a For more information, con- = 

and stored procedures. Additional training covets 
f : S tact GoldenGate Software at = ee eects - 
includes tips for monitoring and trouble- 415-777-0200 pe ll FRCL en 
shooting SQL Server 2005 performance. A ek or atp 


single-user license starts at $499. For more Senge siacliersony GoldenGate Software GoldenGate 9.0 


information, contact KeyStone Learning 


Systems at 800-949-5590 or http://www | scheduled events, now supports Oracle, so | and response actions, graphical reporting, 


-keystonelearning.com. you can manage SQL Server, Windows | and advanced chaining. For more infor- 
Task Scheduler, and Oracle in the same | mation, contact SQL Sentry at 704-895- 
JOB SCHEDULING platform. Event Manager can automate a | 6241, sales@sqlsentry.net, or http://www 
MANAGEMENT logical workflow between SQL Server and | .sqlsentry.net. 
Oracle; the software’s interfz id 
Manage SOL Server and | istic into all scheduled database events 
Oracle Scheduled Events regardless of platform. Event Manager for A 


SQL Sentry announced that SQL Sentry | Oracle features visual schedule manage- Learn How to Implement 
Event Manager, a tool that helps you manage | ment, performance monitoring, alerting | and Maintain a SQL 


LearnKey released Microsoft Certi- 
fied Technology Specialist (MCTS) for 
Structured Query Language (SQL) 2005 
training, a learning track that covers all 
prerequisites for the Microsoft MCTS: 
SQL Server 2005 certification exam. SQL 
Server MVP Wayne Snyder leads you 
through the course material. This learning 
track includes 15 sessions and approxi- 
mately 45 hours of study. LearnKey’s 
MCTS SQL Server 2005 series costs $1080 
for individuals or $2695 for multiple seats. 
The learning track is available online, 
on DVD, and on CD-ROM. For more 
information, contact LearnKey at 800-865- 
0165 or http://wwwilearnkey.com. ED 
SQL Sentry SQL Sentry Event Manager InstantDoc ID 95169 
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Industry Bytes 


New Type of Database 
Vulnerability on the Rise 

Amichai Shulman, CTO of Imperva 
(http://www.imperva.com) and Alan 
Norquist, Imperva’s vice president of mar- 
keting, have alerted us to a new kind of 
vulnerability exploit that database profes- 
sionals are likely to see in upcoming months. 
Shulman, who heads Imperva’s Application 
Defense Center, a research and security 
services center, explained, “Until a year 
ago, most vulnerabilities were related to 
the built-in stored procedures and packages 
supplied with database solutions. But in the 
past year, we've seen a new type of vulner- 
ability related to communication protocols 
between clients and servers. These protocols 
arent exposed to a variety of traffic, but if 
you dig into their implementation, you can 
find vulnerabilities.” 

Imperva’s researchers have seen a lot 
more exploits related to protocol vulner- 
abilities, and Shulman predicts, “This is a 
trend we'll see in the coming year.” He went 
on to stress, “No real workarounds exist yet 
for these kinds of exploits—you can’t fix 
them within the database server.” 

To help database IT pros locate these 
hard-to-track vulnerabilities, Imperva 
released Scuba, a free database-vulnerability 
scanner for SQL Server, Oracle, Sybase, 
and IBM DB2. Scuba scans your database, 
identifies known vulnerabilities and faulty 
or inefficient configurations, and reports the 
overall security status of your database. With 
the analysis in hand, you can decide what 
to do about plugging the holes. The tool is 
a simple download that’s easy to run, and it 
doesn’t use attack techniques to determine 
whether vulnerabilities exist, so it’s safe for 
your network. 

Its important to do periodic security 
scans of your database simply because the 
database is an ever-changing environment. 
“Microsoft has done a great job of locking 
down the default security settings in SQL 
Server,” says Shulman. “But after deploy- 
ment, things change—settings get changed, 


Cultivating Continuing Growth with Continuous Data Protection 

Sonasoft (http://www.sonasoft.com) is taking a unique approach to small-to- 
midsized business (SMB) backup and recovery. In a recent conversation with our 
editors, Sonasoft’s Vice President of Marketing Vas Srinivasan, President and CEO 
Andy Khanna, and Vice President of Engineering and CTO Bilal Ahmed described 
SonaSafe for SQL Server as a “neo-continuous data protection” (CDP) solution. They 
explained that SonaSafe combines replication with backup and recovery functions to 
provide data protection with minimal latency. According to Sonasoft, its competitors 
provide either backup-and-recovery or replication solutions, but no one else provides 
a combined solution. 

Sonasoft has chosen to base SonaSafe for SQL Server on transactional replication, 
which eliminates the need to perform a restore of a full backup in the event of a 
disaster. Instead, administrators can limit restoration efforts to changes that occurred 
after a specific point in time, so the restore can be completed in a matter of minutes 
instead of taking hours or even days. Although SonaSafe’s failover function does 
require some downtime, the time that data is unavailable is minimized. 

SMBs need enterprise-level data protection and availability, Srinivasan says, but can’t 
afford an enterprise CDP solution. Currently, SMBs represent as much as 80 percent 
of the backup-and-recovery market. Sonasoft believes its unique offering puts it in a 
good position to grow the company over the next year. “We don’t aim to compete 
with the enterprise vendors,” says Srinivasan. “We want to get the market share we 
deserve, and we'll earn it.” 

Coupled with Sonasoft’s shift from direct sales to channel distribution of its prod- 
ucts, the release of the latest version of SonaSafe for SQL Server should contribute 
to immediate growth. In addition, in response to ongoing customer feedback, the 
company plans continued upgrades to its products. 

—Dawn Cyr 
InstantDoc ID 94913 


data needs change, people leave the com- 
pany. This tool lets you do continuing assess- 
ments of the database environment so that 
you can stay on top of those changes.’ 

An important point in making the new 
offering freeware, said Norquist, is that 
Scuba is intended to be a starting point that 
helps you see what your next step needs 
to be. “Once you identify a vulnerability, 
you have several paths available to fix it’ If 
you find that you have a vulnerability that 
you can’t get rid of, other Imperva security 
products can help you plug those holes.“For 
example,’ Norquist said, “say you have a 
stored procedure that could allow a user to 
get system admunistrator privileges, but you 
have an application that requires that stored 
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procedure. We provide products that can 
prevent exploits of those vulnerabilities.” 
Imperva will provide ongoing updates 
to the Scuba freeware product. You can 
download Scuba at http://www.imperva 


.com/scuba. SOL] 


—Dawn Cyr 
InstantDoc ID 95019 


Editor's Note: You can view 

all the latest Industry Bytes at 
our new Industry Bytes blog: 
hitp://www.sqimag.com/blog/ 


index.cfmPaction=blogindex& 
departmentid= IO6I. 
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“The ability to maintain 
clean, reliable marketing 
data across multiple capture 
mediums is essential for 
everything we do.” 


- Kevin Parker, Digital Evergreen 
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Key Features 
» Prelocalized runtimes in 7 languages 
b Layered framework engineered for NET 
b Strong typed buniness objects with drag-n-drop 
> Automated pæren -chid relationship management 
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p Integrated localization and tramlaton tools 
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OLE DB 
p intuitive migration from VBE, faxPra, Access 
b Design and deploy SAL databases without sompts 


> Role Based Security add-on embeds security at the 
hase level ef pour StrataFrame Gevelaped application 


WHY StrataFrame? 
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developmen ene 


b Written by experienced application developers 
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b Free technical support Mecrosoft 

b Royalty-free runtime STIT. 
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GD Asua Studio 


There’s Just No Comparison 


Propagate schema changes from Development 
to Staging to Production quickly and efficiently. 
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Development Staging Production 


e Accurate, consistent, 
safe scripts 


e Complete control over 
the comparison and 
synchronization engine 


e Scheduling via Windows 
Scheduler 
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e Easy integration into i | 
your .NET application 


e Full support for SQL 
Server 2005 and 2000 \ 


Toll Free: 
877.777.XSQL (9775) 
sales@x-sql.com 


www.x-SQL.com ‘SQL To download free evaluation copies, visit: 
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Learn Microsoft® SQL Server" 2005 
and Business Intelligence (Bl) 


Introducing the latest in SQL Server 2005 and Business Intelligence (BI) courses from AppDev, the nation’s leader in developer 
learning. Our nationally recognized industry experts will walk you step-by-step through the features and functionalities of these 
exciting SQL Server 2005 technologies! 
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Microsoft SQL Server 2005 20 CD-ROMs or 2 DVD-ROMs 


Microsoft SQL Server 2005 Reporting Services (SSRS) 8 CD-ROMs or 1 DVD-ROM 
Microsoft SQL Server 2005 Analysis Services (SSAS) 8 CD-ROMs or | DVD-ROM 


Microsoft SQL Server 2005 Integration Services (SSIS) 8 CD-ROMs or 1 DVD-ROM 


L00Z Iudy 


i BUY 1, GET 1 


For a limited time, purchase one of our SQL Server 2005 courses above, get another SQL Server 2005 course 
(of equal or lesser value) FREE! Or get all four SQL Server 2005 courses in one money-saving learning suite! 


Visit our Web site today for offer details, plus course outlines and 
AppDev Expert Andy Baron more information about our new SQL Server 2005 courses. 


Same great training, now for your entire team—KSource Online Learning™ °» www.ksourceit.com 


by Michael Otey 


Size Up Team Data 


isual Studio Team Edition for Database Professionals (Team Data) is a long-overdue tool 

from Microsoft that lets database professionals better control and deploy SQL Server 
2005 and 2000 database schema changes. Team Data’s features include database schema 
source control, database object refactoring, database schema and data comparison, and unit 
testing. Part of the Visual Studio Team Edition family, it’s not bundled with SQL Server 2005, 
so you'll need to purchase it separately. Before you take that leap, I'd like to point you to 
some great Microsoft resources where you can find more information about Team Data. 


Videos and Webcasts 

One of the quickest (although not always 
the most exciting) ways to get up to speed 
quickly is to watch technical presentations. 
You can find seven Webcasts and eight 
short videos about Team Data’s features 
at the Database Team Professional Center: 
http://msdn2.microsoft.com/en-us/ 
teamsystem/aa718764.aspx. 


| Microsoft Blogs 

To get an insiders view of Team Data, 
check out these blogs by Microsoft product 
managers (two of whom, Matt Nunn and 
Richard Waymure, also write for SQL Server 
Magazine). 
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e Mairead O’Donovan (http://blogs 
-msdn.com/mairead) 

e Sachin Rekhi (http://blogs.msdn 
.com/sachinre) 

e Cameron Skinner (http://blogs.msdn 
.com/camerons) 

e Richard Waymire (http://blogs.msdn 


.com/rwaymi) 


They offer news about the product, best 
practices, samples, and discussions about 
how companies have deployed Team Data. 
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Team Data FAQs 
Microsofts Team Data Web page provides 
information about the trial version. The site 
also explains features such as version control 
and refactoring: http://msdn2.microsoft 
.com/en-us/teamsystem/aa7 18913.aspx 


Team Data Forum 
‘Another great resource is Microsoft’s Team 
Data MSDN forum, which is frequented 
by many of the product managers. Here, 
you can get answers to many of your ques- 
tions: http://forums.microsoft.com/msdn/ 


showforum.aspx?forumid=725&siteid=1 


Trial Edition 
The best way to learn about a product, of 
‘course, is by trying it. You can download a 
180-day trial version that runs on Windows 
Server 2003 Service Pack 1 (SP1),Windows 
XP SP2, and Windows 2000 SP4. You'll 
also need to have installed either the retail 
or trial edition of Visual Studio 2005 Team 
Suite Edition. Download the trial version 


at http://www.microsoft.com/downloads/ 


details.aspx?familyid=7de00386-893d- 
4142-a778-992b69d482ad&displaylang=en. 
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Real Life 


) SQL Server Superhero 


| STAND TALL. Idera made me a real life SOL Server Superhero. 


TOOLS FOR DATA MANAGEMENT Idera delivers a new generation speed recovery, ease compliance 
of tools for managing the requirements, and dramatically 
world's fastest growing database reduce administrative overhead. 
management system — Microsoft All of Idera’s products are amazingly 
SQL Server. Battle-proven and simple to use, provide remarkable 
Microsoft T, for the enterprise, Idera results, and can be installed in 

GOLD CERTIFIED elps database administrators keep minutes, configured in hours, and 
"Partner SQL Server running at optimum deployed worldwide in days. 
performance, ensure availability, 


